aboutsummaryrefslogtreecommitdiff
path: root/mesonbuild/mtest.py
AgeCommit message (Collapse)AuthorFilesLines
2021-02-10mtest: include classname in <testcase> JUnit elementPaolo Bonzini1-2/+4
It looks like GitLab ignores the suite name and actually uses the classname. Adjust the output accordingly. Fixes: #8316 Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-02-10mtest: cancel stdout/stderr tasks on timeoutPaolo Bonzini1-18/+35
Avoid that the tasks linger and SingleTestRunner.run() never terminates. In order to do this, we need read_decode() and read_decode_lines() to be cancellable, and to handle the CancelledError gracefully while returning the output they have collected so far. For read_decode(), this means always operating on a line-by-line basis, even if console_mode is not ConsoleUser.STDOUT. For read_decode_lines(), instead, we cannot return an iterator. Rather, read_decode_lines() returns the output directly (similar to read_decode) and communication with the parser is mediated by an asyncio.Queue. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-02-10mtest: clean up conditions on whether tests are run in parallelPaolo Bonzini1-10/+13
This makes non-parallel tests emit their output on the fly, similar to ninja console jobs. It also cleans up the code a bit, avoiding the repetition of "self.options.num_processes" tests. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-02-10mtest: fix nits in printing test stdoutPaolo Bonzini1-4/+5
- Ensure the output is terminated with a \n even if the test does not include one. - Ensure that stdout is flushed for each reported result Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-02-09mtest: TestSetup can have [] as an exe_wrapperPaolo Bonzini1-2/+2
Fix "meson test --wrapper foo --setup bar", it should work just fine if the setup does not define a wrapper. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-02-09mtest: run the test output parser as a taskPaolo Bonzini1-5/+6
Start the parsing of the output early; this avoids a deadlock if the test writes to stdout but no one reads from it. It also reports TAP or Rust subtest results as they happen, which was the intention all along. While at it, use a consistent naming conventions for coroutines vs tasks. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-02-09mtest: hide infinite timeout from the progress reportPaolo Bonzini1-3/+8
Avoid printing something like "30/-1s" when tests are run without a timeout or with --timeout-multiplier 0. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-02-02interpreter, mtest: introduce add_test_setup(exclude_suites: ...)Paolo Bonzini1-11/+15
This new keyword argument makes it possible to run specific test setups only on a subset of the tests. For example, to mark some tests as slow and avoid running them by default: add_test_setup('quick', exclude_suites: ['slow'], is_default: true) add_test_setup('slow') It will then be possible to run the slow tests with either `meson test --setup slow` or `meson test --suite slow`.
2021-02-02mtest: extract get_test_setup, rename merge_suite_optionsPaolo Bonzini1-10/+13
merge_suite_options is really about test setups, so rename accordingly. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-02-02mtest: load build data earlyPaolo Bonzini1-23/+21
This will be needed to exclude testsuites from test setups (which are stored in the build data). While at it, since a chdir is needed simplify a bit the loading of tests and benchmarks.
2021-02-02mtest: compute logfile name earlyPaolo Bonzini1-15/+18
Do this before overriding self.options.setup with the default setup.
2021-02-02mtest: remove useless argumentPaolo Bonzini1-3/+3
2021-01-30Merge pull request #8200 from bonzini/mtest-asyncio-logsJussi Pakkanen1-116/+234
mtest: improvements to logging
2021-01-27test: Make timeout <= 0 infinite duractionXavier Claessens1-6/+8
2021-01-22mtest: log individual TAP subtestsPaolo Bonzini1-11/+34
2021-01-22mtest: infrastructure to print subtest resultsPaolo Bonzini1-7/+18
2021-01-22mtest: move --print-errorlogs output during the test runPaolo Bonzini1-18/+13
Print the (shortened) output of the failed tests as they happen. If neither --verbose nor --print-errorlogs was specified, omit the summary of failures, because it is pretty much the same as the earlier output of "meson test".
2021-01-22mtest: log test start in verbose modePaolo Bonzini1-13/+37
In non-parallel verbose mode the output of the test/benchmark is not buffered, therefore the command line is only printed by ConsoleLogger for failing tests and only after the test has run. Verbose mode is designed mostly for CI systems, where output must be human readable but is generally consumed from a browser with "Find" commands rather than from a terminal. With this usecase in mind, it is better to provide as much detail as possible, so add more output and just tell the user which tests have started. Do so, using the recently introduced TestResult.RUNNING state.
2021-01-22mtest: call TestLogger.start_test when TestRun is in the RUNNING statePaolo Bonzini1-8/+13
2021-01-22mtest: use buffered stdout/stderr in parallel modePaolo Bonzini1-2/+5
Similar to ninja, buffer stdout/stderr even in verbose mode if more than one test is being run in parallel.
2021-01-22mtest: quote environment variable values and command line argumentsPaolo Bonzini1-2/+9
2021-01-22mtest: include full environment in the TestRun objectPaolo Bonzini1-32/+34
Ensure that all the required modifications are included in the logs. This makes it possible for users to cut-and-paste from the logs when trying to reproduce failures outside Meson.
2021-01-22mtest: make log output more suitable for consolePaolo Bonzini1-37/+74
Right now the same code is used to print the logs for both the console and the text log. Differentiating them lets the important bits of the console output stand out, and makes the console output a bit more readable.
2021-01-22mtest: small refactoring of log printingPaolo Bonzini1-23/+34
Start moving console-specific code to ConsoleLogger, as well as moving code out of get_log().
2021-01-22mtest: log in verbose mode, but not in gdb modePaolo Bonzini1-1/+1
This is now possible because all stdout/stderr goes through asyncio pipes. However, logs make little sense in gdb mode.
2021-01-20mtest: collect stdout/stderr even in verbose modePaolo Bonzini1-14/+27
Using verbose mode dropped stdout/stderr from the logs, because it was not captured. Now that we can easily stick code in the middle of the reading of stdout/stderr, use that to print stdout and stderr on the fly while also capturing them for the logs. The output is line-buffered. As a side effect, this also fixes a possible deadlock due to not using ensure_future around stdo_task and stde_task. In particular: - the stdo_task coroutine would not terminate until the test closed stdo_task - the stde_task coroutine would not start until the stdo_task coroutine finished Therefore, the test could get stuck waiting for its parent to read the contents of stderr, but that would not happen because Meson was still in the stdo_task coroutine.
2021-01-15mtest: move I/O handling to TestSubprocessPaolo Bonzini1-36/+44
Move the logic to start the read/decode tasks to TestSubprocess and keep SingleTestRunner simple. The lines() inner function is tweaked to produce stdout as a future. This removes the nonlocal access (which is not possible anymore when the code is moved out of _run_cmd), and also lets _run_cmd use "await stdo_task" for both parsed and unparsed output.
2021-01-15mtest: tweak the gathering of stdo_task/stde_task resultsPaolo Bonzini1-9/+14
After the next patch, we will need to complete parse_task before stdo_task (because parse_task will not set the "stdo" variable anymore but it will still collect stdout just like now). Do the change now to isolate the more complicated changes.
2021-01-15mtest: disable the progress report in gdb modePaolo Bonzini1-3/+5
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-01-15mtest: introduce ConsoleUserPaolo Bonzini1-2/+22
This new enum can be used by TestSubprocess and TestHarness to understand (at a higher level) how SingleTestRunner sets up stdout/stderr redirection.
2021-01-15mtest: turn TestRun into a hierarchyPaolo Bonzini1-91/+128
Make SimpleTestRunner oblivious of the various test protocols. All the different "complete_*" and "parse_*" methods move to the subclasses of TestRun.
2021-01-15mtest: pass command line to TestRun.startPaolo Bonzini1-18/+16
The command line is already available when the test is started. Pass it to TestRun.start instead of TestRun.complete*.
2021-01-15mtest: add complete_skip to TestRunPaolo Bonzini1-1/+4
2021-01-15mtest: simplify complete_exitcodePaolo Bonzini1-8/+5
There is no need anymore to pass the JUnit XML down to complete_exitcode. Just set self.junit in complete_gtest instead.
2021-01-15mtest: store the environment in the TestRunPaolo Bonzini1-1/+1
The test_env was lost from the TestRun object in commit 30741a0f2 ("mtest: create TestRun object early on"). Fix things.
2021-01-15mtest: do not print time out message twicePaolo Bonzini1-2/+0
Rebase mistake in "mtest: move timeout message to ConsoleLogger".
2021-01-14mtest: allow quickly interrupting the test runPaolo Bonzini1-5/+17
The new behavior of interrupting the longest running test with Ctrl-C is useful when tests hang, but not when the run is completely broken for some reason. Psychology tells us that the user will compulsively spam Ctrl-C in this case, so exit if three Ctrl-C's are detected within a second.
2021-01-14mtest: fix flake8Paolo Bonzini1-2/+2
2021-01-07mtest: print TAP subtest countPaolo Bonzini1-2/+15
The parentheses look ugly in the progress report. To keep it aligned with the test outcomes, remove them from the outcomes as well.
2021-01-07mtest: print time that the test has been runningPaolo Bonzini1-4/+18
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-01-07mtest: align decimal point of test durationsPaolo Bonzini1-2/+9
2021-01-07mtest: create runners in advancePaolo Bonzini1-17/+24
Compute all options in advance so that we can compute the maximum timeout.
2021-01-07mtest: move timeout message to ConsoleLoggerPaolo Bonzini1-0/+4
This adds a point where to call the progress report flush() method. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-01-07mtest: store timeout in TestRunPaolo Bonzini1-13/+13
This will be useful when printing the progress report. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-01-07mtest: add more formatting options to TestHarness.formatPaolo Bonzini1-15/+34
Allow leaving extra space in the left column, as well as customizing parts of the printed line. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-01-07mtest: align correctly tests with wide Unicode charactersPaolo Bonzini1-3/+13
This correctly formats tests with CJK names or, well, emoji. It is not perfect (for example it does not correctly format emoji that are variations of 1-wide characters), but it is as good as most terminal emulators. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-01-07mtest: make test output parsing asynchronousPaolo Bonzini1-22/+34
Instead of slurping in the entire stream, build the TestResult along the way. This allows reporting the results of TAP and Rust subtests as they come in, either as part of the progress report or (in the future) as individual lines of the output.
2021-01-07mtest: move Rust parsing inside TestRunPaolo Bonzini1-41/+40
Make the code look like the TAP parser. This simplifies the introduction of asynchronous parsing.
2021-01-07mtest: read test stdout/stderr via asyncio pipesPaolo Bonzini1-21/+29
Instead of creating temporary files, get the StreamReaders from _run_subprocess's returned object. Through asyncio magic, their contents will be read as it becomes ready and then returned when the StreamReader.read future is awaited. Because of this change, the stdout and stderr can be easily preserved when TestSubprocess returns an additional_error. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-01-07mtest: do not wait inside _run_subprocessPaolo Bonzini1-66/+81
We would like SingleTestRunner to run code before waiting on the process, for example starting tasks to read stdout and stderr. Return a new object that is able to complete _run_subprocess's task. In the next patch, SingleTestRunner will also use the object to get hold of the stdout and stderr StreamReaders. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
generated by cgit v1.1 at 2025-07-14 04:54:02 +0000