qemu/python/avocado.cfg

[run]
test_runner = nrunner

[simpletests]
# Don't show stdout/stderr in the test *summary*
status.failure_fields = ['status']

[job]
# Don't show the full debug.log output; only select stdout/stderr.
output.testlogs.logfiles = ['stdout', 'stderr']

# Show full stdout/stderr only on tests that FAIL
output.testlogs.statuses = ['FAIL']
python/aqmp: Add Coverage.py support I'm not exposing this via the Makefile help, it's not likely to be useful to passersby. Switch the avocado runner to the 'legacy' runner for now, as the new runner seems to obscure coverage reports, again. Usage is to enter your venv of choice and then: `make check-coverage && xdg-open htmlcov/index.html`. Signed-off-by: John Snow <jsnow@redhat.com> Message-id: 20210915162955.333025-28-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com> 2021-09-15 12:29:55 -04:00			`[run]`
python: use avocado's "new" runner The old legacy runner no longer seems to work with output logging, so we can't see failure logs when a test case fails. The new runner doesn't (seem to) support Coverage.py yet, but seeing error output is a more important feature. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Beraldo Leal <bleal@redhat.com> Message-id: 20220119193916.4138217-3-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com> 2022-01-19 14:39:13 -05:00			`test_runner = nrunner`
python/aqmp: Add Coverage.py support I'm not exposing this via the Makefile help, it's not likely to be useful to passersby. Switch the avocado runner to the 'legacy' runner for now, as the new runner seems to obscure coverage reports, again. Usage is to enter your venv of choice and then: `make check-coverage && xdg-open htmlcov/index.html`. Signed-off-by: John Snow <jsnow@redhat.com> Message-id: 20210915162955.333025-28-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com> 2021-09-15 12:29:55 -04:00
python: add avocado-framework and tests Try using avocado to manage our various tests; even though right now they're only invoking shell scripts and not really running any python-native code. Create tests/, and add shell scripts which call out to mypy, flake8, pylint and isort to enforce the standards in this directory. Add avocado-framework to the setup.cfg development dependencies, and add avocado.cfg to store some preferences for how we'd like the test output to look. Finally, add avocado-framework to the Pipfile environment and lock the new dependencies. We are using avocado >= 87.0 here to take advantage of some features that Cleber has helpfully added to make the test output here very friendly and easy to read for developers that might chance upon the output in Gitlab CI. [Note: ALL of the dependencies get updated to the most modern versions that exist at the time of this writing. No way around it that I have seen. Not ideal, but so it goes.] Provided you have the right development dependencies (mypy, flake8, isort, pylint, and now avocado-framework) You should be able to run "avocado --config avocado.cfg run tests/" from the python folder to run all of these linters with the correct arguments. (A forthcoming commit adds the much easier 'make check'.) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-28-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com> 2021-05-27 17:17:11 -04:00			`[simpletests]`
			`# Don't show stdout/stderr in the test summary`
			`status.failure_fields = ['status']`

			`[job]`
			`# Don't show the full debug.log output; only select stdout/stderr.`
			`output.testlogs.logfiles = ['stdout', 'stderr']`

			`# Show full stdout/stderr only on tests that FAIL`
			`output.testlogs.statuses = ['FAIL']`