diff options
Diffstat (limited to 'docs/markdown/Unit-tests.md')
| -rw-r--r-- | docs/markdown/Unit-tests.md | 169 |
1 files changed, 89 insertions, 80 deletions
diff --git a/docs/markdown/Unit-tests.md b/docs/markdown/Unit-tests.md index 60fcad2..92014bb 100644 --- a/docs/markdown/Unit-tests.md +++ b/docs/markdown/Unit-tests.md @@ -4,8 +4,8 @@ short-description: Meson's own unit-test system # Unit tests -Meson comes with a fully functional unit test system. To use it simply build -an executable and then use it in a test. +Meson comes with a fully functional unit test system. To use it simply +build an executable and then use it in a test. ```meson e = executable('prog', 'testprog.c') @@ -40,17 +40,19 @@ can be disabled as discussed in [test()](Reference-manual.md#test). ## Coverage -If you enable coverage measurements by giving Meson the command line flag -`-Db_coverage=true`, you can generate coverage reports after running the -tests (running the tests is required to gather the list of functions that get -called). Meson will autodetect what coverage generator tools you have -installed and will generate the corresponding targets. These targets are -`coverage-xml` and `coverage-text` which are both provided by -[Gcovr](http://gcovr.com) (version 3.3 or higher) and `coverage-html`, which -requires [Lcov](https://ltp.sourceforge.io/coverage/lcov.php) and -[GenHTML](https://linux.die.net/man/1/genhtml) or [Gcovr](http://gcovr.com). -As a convenience, a high-level `coverage` target is also generated which will -produce all 3 coverage report types, if possible. +If you enable coverage measurements by giving Meson the command line +flag `-Db_coverage=true`, you can generate coverage reports after +running the tests (running the tests is required to gather the list of +functions that get called). Meson will autodetect what coverage +generator tools you have installed and will generate the corresponding +targets. These targets are `coverage-xml` and `coverage-text` which +are both provided by [Gcovr](http://gcovr.com) (version 3.3 or higher) +and `coverage-html`, which requires +[Lcov](https://ltp.sourceforge.io/coverage/lcov.php) and +[GenHTML](https://linux.die.net/man/1/genhtml) or +[Gcovr](http://gcovr.com). As a convenience, a high-level `coverage` +target is also generated which will produce all 3 coverage report +types, if possible. The output of these commands is written to the log directory `meson-logs` in your build directory. @@ -58,21 +60,23 @@ your build directory. ## Parallelism To reduce test times, Meson will by default run multiple unit tests in -parallel. It is common to have some tests which can not be run in parallel -because they require unique hold on some resource such as a file or a D-Bus -name. You have to specify these tests with a keyword argument. +parallel. It is common to have some tests which can not be run in +parallel because they require unique hold on some resource such as a +file or a D-Bus name. You have to specify these tests with a keyword +argument. ```meson test('unique test', t, is_parallel : false) ``` -Meson will then make sure that no other unit test is running at the same -time. Non-parallel tests take longer to run so it is recommended that you -write your unit tests to be parallel executable whenever possible. +Meson will then make sure that no other unit test is running at the +same time. Non-parallel tests take longer to run so it is recommended +that you write your unit tests to be parallel executable whenever +possible. -By default Meson uses as many concurrent processes as there are cores on the -test machine. You can override this with the environment variable -`MESON_TESTTHREADS` like this. +By default Meson uses as many concurrent processes as there are cores +on the test machine. You can override this with the environment +variable `MESON_TESTTHREADS` like this. ```console $ MESON_TESTTHREADS=5 meson test @@ -82,10 +86,10 @@ $ MESON_TESTTHREADS=5 meson test *(added in version 0.52.0)* -Tests can be assigned a priority that determines when a test is *started*. -Tests with higher priority are started first, tests with lower priority -started later. The default priority is 0, meson makes no guarantee on the -ordering of tests with identical priority. +Tests can be assigned a priority that determines when a test is +*started*. Tests with higher priority are started first, tests with +lower priority started later. The default priority is 0, meson makes +no guarantee on the ordering of tests with identical priority. ```meson test('started second', t, priority : 0) @@ -93,34 +97,35 @@ test('started third', t, priority : -50) test('started first', t, priority : 1000) ``` -Note that the test priority only affects the starting order of tests and -subsequent tests are affected by how long it takes previous tests to -complete. It is thus possible that a higher-priority test is still running -when lower-priority tests with a shorter runtime have completed. +Note that the test priority only affects the starting order of tests +and subsequent tests are affected by how long it takes previous tests +to complete. It is thus possible that a higher-priority test is still +running when lower-priority tests with a shorter runtime have +completed. ## Skipped tests and hard errors Sometimes a test can only determine at runtime that it can not be run. -For the default `exitcode` testing protocol, the GNU standard approach in -this case is to exit the program with error code 77. Meson will detect this -and report these tests as skipped rather than failed. This behavior was added -in version 0.37.0. +For the default `exitcode` testing protocol, the GNU standard approach +in this case is to exit the program with error code 77. Meson will +detect this and report these tests as skipped rather than failed. This +behavior was added in version 0.37.0. -For TAP-based tests, skipped tests should print a single line starting with -`1..0 # SKIP`. +For TAP-based tests, skipped tests should print a single line starting +with `1..0 # SKIP`. -In addition, sometimes a test fails set up so that it should fail even if it -is marked as an expected failure. The GNU standard approach in this case is -to exit the program with error code 99. Again, Meson will detect this and -report these tests as `ERROR`, ignoring the setting of `should_fail`. This -behavior was added in version 0.50.0. +In addition, sometimes a test fails set up so that it should fail even +if it is marked as an expected failure. The GNU standard approach in +this case is to exit the program with error code 99. Again, Meson will +detect this and report these tests as `ERROR`, ignoring the setting of +`should_fail`. This behavior was added in version 0.50.0. ## Testing tool -The goal of the meson test tool is to provide a simple way to run tests in a -variety of different ways. The tool is designed to be run in the build -directory. +The goal of the meson test tool is to provide a simple way to run +tests in a variety of different ways. The tool is designed to be run +in the build directory. The simplest thing to do is just to run all tests. @@ -162,9 +167,10 @@ Multiple suites are specified like: $ meson test --suite foo --suite bar ``` -NOTE: If you choose to specify both suite(s) and specific test name(s), the -test name(s) must be contained in the suite(s). This however is redundant-- -it would be more useful to specify either specific test names or suite(s). +NOTE: If you choose to specify both suite(s) and specific test +name(s), the test name(s) must be contained in the suite(s). This +however is redundant-- it would be more useful to specify either +specific test names or suite(s). ### Other test options @@ -193,25 +199,26 @@ Meson also supports running the tests under GDB. Just doing this: $ meson test --gdb testname ``` -Meson will launch `gdb` all set up to run the test. Just type `run` in the -GDB command prompt to start the program. +Meson will launch `gdb` all set up to run the test. Just type `run` in +the GDB command prompt to start the program. -The second use case is a test that segfaults only rarely. In this case you -can invoke the following command: +The second use case is a test that segfaults only rarely. In this case +you can invoke the following command: ```console $ meson test --gdb --repeat=10000 testname ``` -This runs the test up to 10 000 times under GDB automatically. If the program -crashes, GDB will halt and the user can debug the application. Note that -testing timeouts are disabled in this case so `meson test` will not kill -`gdb` while the developer is still debugging it. The downside is that if the -test binary freezes, the test runner will wait forever. +This runs the test up to 10 000 times under GDB automatically. If the +program crashes, GDB will halt and the user can debug the application. +Note that testing timeouts are disabled in this case so `meson test` +will not kill `gdb` while the developer is still debugging it. The +downside is that if the test binary freezes, the test runner will wait +forever. -Sometimes, the GDB binary is not in the PATH variable or the user wants to -use a GDB replacement. Therefore, the invoked GDB program can be specified -*(added 0.52.0)*: +Sometimes, the GDB binary is not in the PATH variable or the user +wants to use a GDB replacement. Therefore, the invoked GDB program can +be specified *(added 0.52.0)*: ```console $ meson test --gdb --gdb-path /path/to/gdb testname @@ -221,41 +228,43 @@ $ meson test --gdb --gdb-path /path/to/gdb testname $ meson test --print-errorlogs ``` -Meson will report the output produced by the failing tests along with other -useful information as the environmental variables. This is useful, for -example, when you run the tests on Travis-CI, Jenkins and the like. +Meson will report the output produced by the failing tests along with +other useful information as the environmental variables. This is +useful, for example, when you run the tests on Travis-CI, Jenkins and +the like. -For further information see the command line help of Meson by running `meson -test -h`. +For further information see the command line help of Meson by running +`meson test -h`. ## Legacy notes -If `meson test` does not work for you, you likely have a old version of -Meson. In that case you should call `mesontest` instead. If `mesontest` -doesn't work either you have a very old version prior to 0.37.0 and should -upgrade. +If `meson test` does not work for you, you likely have a old version +of Meson. In that case you should call `mesontest` instead. If +`mesontest` doesn't work either you have a very old version prior to +0.37.0 and should upgrade. ## Test outputs -Meson will write several different files with detailed results of running -tests. These will be written into $builddir/meson-logs/ +Meson will write several different files with detailed results of +running tests. These will be written into $builddir/meson-logs/ ### testlog.json -This is not a proper json file, but a file containing one valid json object -per line. This is file is designed so each line is streamed out as each test -is run, so it can be read as a stream while the test harness is running +This is not a proper json file, but a file containing one valid json +object per line. This is file is designed so each line is streamed out +as each test is run, so it can be read as a stream while the test +harness is running ### testlog.junit.xml -This is a valid JUnit XML description of all tests run. It is not streamed -out, and is written only once all tests complete running. +This is a valid JUnit XML description of all tests run. It is not +streamed out, and is written only once all tests complete running. -When tests use the `tap` protocol each test will be recorded as a testsuite -container, with each case named by the number of the result. +When tests use the `tap` protocol each test will be recorded as a +testsuite container, with each case named by the number of the result. -When tests use the `gtest` protocol meson will inject arguments to the test -to generate it's own JUnit XML, which meson will include as part of this XML -file. +When tests use the `gtest` protocol meson will inject arguments to the +test to generate it's own JUnit XML, which meson will include as part +of this XML file. *New in 0.55.0* |