aboutsummaryrefslogtreecommitdiff
path: root/docs/devel
diff options
context:
space:
mode:
authorStefan Hajnoczi <stefanha@redhat.com>2023-12-04 08:03:18 -0500
committerStefan Hajnoczi <stefanha@redhat.com>2023-12-04 08:03:18 -0500
commit17dacf7ac9e2f076c15f32a290203f8f571a8800 (patch)
tree8d9f78a3d38561ec1224f25504d5de272ef5bdfc /docs/devel
parent173e8280643ad3b7221b12a8deafce826f5e7efb (diff)
parent5d25fcb702406286a17806dd53d10eec1d2cc34f (diff)
downloadqemu-17dacf7ac9e2f076c15f32a290203f8f571a8800.zip
qemu-17dacf7ac9e2f076c15f32a290203f8f571a8800.tar.gz
qemu-17dacf7ac9e2f076c15f32a290203f8f571a8800.tar.bz2
Merge tag 'pull-more-8.2-fixes-011223-2' of https://gitlab.com/stsquad/qemu into staging
Flaky avocado tests, gdbstub and gitlab tweaks - gdbstub, properly halt when QEMU is having IO issues - convert skipIf(GITLAB_CI) to skipUnless(QEMU_TEST_FLAKY_TESTS) - tag sbsa-ref tests as TCG only - build the correct microblaze for avocado-system-ubuntu - add optional flaky tests job to CI # -----BEGIN PGP SIGNATURE----- # # iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmVqHFgACgkQ+9DbCVqe # KkQHLwgAjP2iL5LSa3FaMUoESJQqRB0rpoJ80gtEtmvmgRF0fHsRfHtDdMN9h2Ed # YilCDhMKLyr2ZoK4atyuc5SR6vCXI5RAvfTddex0xSxlvBX5Z5+1FMC6yA8SDJM7 # ezEXACEKHiGv+l8gvOZOf9ZYEgh8DMJYFMbrtxuxKWw/kAjZ3R3X/ChCL94ZCPRe # 486wqPIQfp5EPs2ddsW4DYFTjLpK5ImX+u/5kdaEGXwcg8UoLmQ9BVIrN/hYJ6u5 # t/mAp1qVIQwSOSUBnerQ4ZkVQfCgLtEtiDtt8EZjUbQD3DcLjfHFjTwVlpqcC1zs # wHXYpLbD5jkthqav5E0DObCF9gIZdA== # =qtvU # -----END PGP SIGNATURE----- # gpg: Signature made Fri 01 Dec 2023 12:48:08 EST # gpg: using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44 # gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [unknown] # gpg: WARNING: This key is not certified with a trusted signature! # gpg: There is no indication that the signature belongs to the owner. # Primary key fingerprint: 6685 AE99 E751 67BC AFC8 DF35 FBD0 DB09 5A9E 2A44 * tag 'pull-more-8.2-fixes-011223-2' of https://gitlab.com/stsquad/qemu: gitlab: add optional job to run flaky avocado tests gitlab: build the correct microblaze target tests/avocado: tag sbsa tests as tcg only docs/devel: rationalise unstable gitlab tests under FLAKY_TESTS gdbstub: use a better signal when we halt for IO reasons Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Diffstat (limited to 'docs/devel')
-rw-r--r--docs/devel/testing.rst32
1 files changed, 21 insertions, 11 deletions
diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index 22218db..bd13230 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -1371,23 +1371,33 @@ conditions. For example, tests that take longer to execute when QEMU is
compiled with debug flags. Therefore, the ``AVOCADO_TIMEOUT_EXPECTED`` variable
has been used to determine whether those tests should run or not.
-GITLAB_CI
-^^^^^^^^^
-A number of tests are flagged to not run on the GitLab CI. Usually because
-they proved to the flaky or there are constraints on the CI environment which
-would make them fail. If you encounter a similar situation then use that
-variable as shown on the code snippet below to skip the test:
+QEMU_TEST_FLAKY_TESTS
+^^^^^^^^^^^^^^^^^^^^^
+Some tests are not working reliably and thus are disabled by default.
+This includes tests that don't run reliably on GitLab's CI which
+usually expose real issues that are rarely seen on developer machines
+due to the constraints of the CI environment. If you encounter a
+similar situation then raise a bug and then mark the test as shown on
+the code snippet below:
.. code::
- @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
+ # See https://gitlab.com/qemu-project/qemu/-/issues/nnnn
+ @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on GitLab')
def test(self):
do_something()
-QEMU_TEST_FLAKY_TESTS
-^^^^^^^^^^^^^^^^^^^^^
-Some tests are not working reliably and thus are disabled by default.
-Set this environment variable to enable them.
+You can also add ``:avocado: tags=flaky`` to the test meta-data so
+only the flaky tests can be run as a group:
+
+.. code::
+
+ env QEMU_TEST_FLAKY_TESTS=1 ./pyvenv/bin/avocado \
+ run tests/avocado -filter-by-tags=flaky
+
+Tests should not live in this state forever and should either be fixed
+or eventually removed.
+
Uninstalling Avocado
~~~~~~~~~~~~~~~~~~~~