diff options
author | Stefan Hajnoczi <stefanha@redhat.com> | 2023-12-04 08:03:18 -0500 |
---|---|---|
committer | Stefan Hajnoczi <stefanha@redhat.com> | 2023-12-04 08:03:18 -0500 |
commit | 17dacf7ac9e2f076c15f32a290203f8f571a8800 (patch) | |
tree | 8d9f78a3d38561ec1224f25504d5de272ef5bdfc /docs/devel | |
parent | 173e8280643ad3b7221b12a8deafce826f5e7efb (diff) | |
parent | 5d25fcb702406286a17806dd53d10eec1d2cc34f (diff) | |
download | qemu-17dacf7ac9e2f076c15f32a290203f8f571a8800.zip qemu-17dacf7ac9e2f076c15f32a290203f8f571a8800.tar.gz qemu-17dacf7ac9e2f076c15f32a290203f8f571a8800.tar.bz2 |
Merge tag 'pull-more-8.2-fixes-011223-2' of https://gitlab.com/stsquad/qemu into staging
Flaky avocado tests, gdbstub and gitlab tweaks
- gdbstub, properly halt when QEMU is having IO issues
- convert skipIf(GITLAB_CI) to skipUnless(QEMU_TEST_FLAKY_TESTS)
- tag sbsa-ref tests as TCG only
- build the correct microblaze for avocado-system-ubuntu
- add optional flaky tests job to CI
# -----BEGIN PGP SIGNATURE-----
#
# iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmVqHFgACgkQ+9DbCVqe
# KkQHLwgAjP2iL5LSa3FaMUoESJQqRB0rpoJ80gtEtmvmgRF0fHsRfHtDdMN9h2Ed
# YilCDhMKLyr2ZoK4atyuc5SR6vCXI5RAvfTddex0xSxlvBX5Z5+1FMC6yA8SDJM7
# ezEXACEKHiGv+l8gvOZOf9ZYEgh8DMJYFMbrtxuxKWw/kAjZ3R3X/ChCL94ZCPRe
# 486wqPIQfp5EPs2ddsW4DYFTjLpK5ImX+u/5kdaEGXwcg8UoLmQ9BVIrN/hYJ6u5
# t/mAp1qVIQwSOSUBnerQ4ZkVQfCgLtEtiDtt8EZjUbQD3DcLjfHFjTwVlpqcC1zs
# wHXYpLbD5jkthqav5E0DObCF9gIZdA==
# =qtvU
# -----END PGP SIGNATURE-----
# gpg: Signature made Fri 01 Dec 2023 12:48:08 EST
# gpg: using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8 DF35 FBD0 DB09 5A9E 2A44
* tag 'pull-more-8.2-fixes-011223-2' of https://gitlab.com/stsquad/qemu:
gitlab: add optional job to run flaky avocado tests
gitlab: build the correct microblaze target
tests/avocado: tag sbsa tests as tcg only
docs/devel: rationalise unstable gitlab tests under FLAKY_TESTS
gdbstub: use a better signal when we halt for IO reasons
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Diffstat (limited to 'docs/devel')
-rw-r--r-- | docs/devel/testing.rst | 32 |
1 files changed, 21 insertions, 11 deletions
diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst index 22218db..bd13230 100644 --- a/docs/devel/testing.rst +++ b/docs/devel/testing.rst @@ -1371,23 +1371,33 @@ conditions. For example, tests that take longer to execute when QEMU is compiled with debug flags. Therefore, the ``AVOCADO_TIMEOUT_EXPECTED`` variable has been used to determine whether those tests should run or not. -GITLAB_CI -^^^^^^^^^ -A number of tests are flagged to not run on the GitLab CI. Usually because -they proved to the flaky or there are constraints on the CI environment which -would make them fail. If you encounter a similar situation then use that -variable as shown on the code snippet below to skip the test: +QEMU_TEST_FLAKY_TESTS +^^^^^^^^^^^^^^^^^^^^^ +Some tests are not working reliably and thus are disabled by default. +This includes tests that don't run reliably on GitLab's CI which +usually expose real issues that are rarely seen on developer machines +due to the constraints of the CI environment. If you encounter a +similar situation then raise a bug and then mark the test as shown on +the code snippet below: .. code:: - @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab') + # See https://gitlab.com/qemu-project/qemu/-/issues/nnnn + @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on GitLab') def test(self): do_something() -QEMU_TEST_FLAKY_TESTS -^^^^^^^^^^^^^^^^^^^^^ -Some tests are not working reliably and thus are disabled by default. -Set this environment variable to enable them. +You can also add ``:avocado: tags=flaky`` to the test meta-data so +only the flaky tests can be run as a group: + +.. code:: + + env QEMU_TEST_FLAKY_TESTS=1 ./pyvenv/bin/avocado \ + run tests/avocado -filter-by-tags=flaky + +Tests should not live in this state forever and should either be fixed +or eventually removed. + Uninstalling Avocado ~~~~~~~~~~~~~~~~~~~~ |