aboutsummaryrefslogtreecommitdiff
path: root/mesonbuild/mtest.py
AgeCommit message (Collapse)AuthorFilesLines
2020-10-28Fix gtest invoking while workdir is set (#7904)Sergey Kartashev1-3/+3
* Fix gtest invoking while workdir is set * Fix gtest invoking when workdir is not set * Code style fix Co-authored-by: Sergey Kartashev <kartashev.sv@mipt.ru>
2020-10-13mtest: Allow filtering tests by subprojectNirbheek Chauhan1-3/+26
You could always specify a list of tests to run by passing the names as arguments to `meson test`. If there were multiple tests with that name (in the same project or different subprojects), all of them would be run. Now you can: 1. Run all tests with the specified name from a specific subproject: `meson test subprojname:testname` 1. Run all tests defined in a specific subproject: `meson test subprojectname:` Also forbid ':' in test names. We already forbid this elsewhere, so should not be a big deal.
2020-10-12mtest: collect failures regardless of colorized consoleCamilo Celis Guzman1-9/+11
2020-10-04pathlib: Fix resolve() by overriding it in Python 3.5Daniel Mensinger1-1/+1
2020-09-10mtest: Refactor test data checkingNirbheek Chauhan1-21/+12
2020-09-10mtest: Check version in the test data after loadingNirbheek Chauhan1-10/+27
Same as coredata.dat and build.dat loading. Also, do not assert if things change. Raise the appropriate exception. Fixes https://github.com/mesonbuild/meson/issues/7613
2020-09-08typing: more fixesDaniel Mensinger1-32/+35
2020-09-08typing: fully annotate mintro and mtestDaniel Mensinger1-13/+13
2020-09-04environment: use ExternalProgram to find ninjaPaolo Bonzini1-1/+1
This allows the NINJA environment variable to support all the Windows special cases, especially allowing an absolute path without extension. Based on a patch by Yonggang Luo. Fixes: #7659 Suggested-by: Nirbheek Chauhan <nirbheek@centricular.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-08-30Be stricter when detecting Windows/CygwinChristoph Reiter1-3/+2
This removes the check for "mingw" for platform.system(). The only case I know where "mingw" is return is if using a msys Python under a msys2 mingw environment. This combination is not really supported by meson and will result in weird errors, so remove the check. The second change is checking sys.platform for cygwin instead of platform.system(). The former is document to return "cygwin", while the latter is not and just returns uname(). While under Cygwin it uname() always starts with "cygwin" it's not hardcoded in MSYS2 and starts with the environment name. Using sys.platform is safer here. Fixes #7552
2020-08-13mtest: Actually call colorize_console()Nirbheek Chauhan1-1/+1
2020-08-08mtest: fix skipping with various prefixesMarc-André Lureau1-1/+1
According to the specification: https://testanything.org/tap-specification.html#skipping-tests The harness should report the text after # SKIP\S*\s+ as a reason for skipping. (it's not exactly like the TODO directive, the phrasing/presentation of the spec could be improved).
2020-08-05mtest: TestResult.SKIP is not a failure (#7525)Simon McVittie1-1/+1
* mtest: TestResult.SKIP is not a failure If some but not all tests in a run were skipped, then the overall result is given by whether there were any failures among the non-skipped tests. Resolves: https://github.com/mesonbuild/meson/issues/7515 Signed-off-by: Simon McVittie <smcv@debian.org> * Add test-cases for partially skipped TAP tests issue7515.txt is the output of one of the real TAP tests in gjs, which failed as a result of #7515. The version inline in meson.build is a minimal reproducer. Signed-off-by: Simon McVittie <smcv@debian.org>
2020-07-30mtest: report failures and overall summary *after* test(s) logsCamilo Celis Guzman1-2/+8
2020-05-18pass exe_wrapper to test scripts through the environmentDylan Baker1-15/+17
This adds a new MESON_EXE_WRAPPER environment variable containing the string form of the exe_wrapper, if there is an exe_wrapper defined. Fixes #4427
2020-05-18backends: Consider arguments passed to a test when cross compilingDylan Baker1-3/+4
Otherwise a wrapper script which takes an executable as an argument will mistakenly run when that executable is cross compiled. This does not wrap said executable in an exe_wrapper, just skip it. Fixes #5982
2020-05-18mtest: don't use len() to test container emptinessDylan Baker1-5/+5
It's not idiomatic python and is significantly slower than not using the bool protocol.
2020-05-04Add native support for gtest testsDylan Baker1-7/+47
Gtest can output junit results with a command line switch. We can parse this to get more detailed results than the returncode, and put those in our own Junit output. We basically just throw away the top level 'testsuites' object, then fixup the names of the tests, and shove that into our junit.
2020-04-30Convert test protocol into an enumDylan Baker1-2/+3
This gives us better type safety, and will be important as we add more test methods
2020-04-30mtest: use argparse.type to simplify some codeDylan Baker1-2/+1
2020-04-30mtest: Replace if (bool) { return bool; } with return bool;Dylan Baker1-3/+1
2020-04-23mtest: Generate a JUnit xml result fileDylan Baker1-0/+113
JUnit is pretty ubiquitous, lots of services and results viewers understand it, in particular gitlab and jenkins know how to consume JUnit xml. This means projects using CI services can have their test results consumed automatically. Fixes: #6972
2020-04-23mtest: Store individual results for TAP testsDylan Baker1-21/+23
This will be used by the Junit writer
2020-04-22mtest: Use textrwap.dedent for large blockDylan Baker1-8/+9
This allows editors like vim to properly fold the code, and makes it generally easier to read.
2020-04-03mtest: Improve test output formatting.Hemmo Nieminen1-22/+29
Remove some weirdness from test output such as extra commas, missing spaces and way too precise time durations. Also improve the overall alignment of the output.
2020-03-19mtest: terminate a test via SIGTERM first then (if needed) via SIGKILLCamilo Celis Guzman1-7/+20
2020-03-03Cherry-picking - (rm python2 % add python3 .format) (#6725)Michael Brockus1-30/+28
2020-01-08types: import typing as T (fixes #6333)Daniel Mensinger1-42/+42
2019-11-07mesonbuild/mtest: plumb and report a test's start timeCamilo Celis Guzman1-11/+17
2019-11-07CI: add initial type annotation checkingMichael Hirsch, Ph.D1-3/+3
2019-11-06Fix typos found by codespellWolfgang Stöggl1-1/+1
- Typos were found by codespell v1.16.0
2019-10-04mtest: add gdb_path optionGerion Entrup1-1/+3
When using the '--gdb' argument of meson test the executed binary can now be specified with '--gdb-path'. Closing: #4373
2019-09-28mtest: Don't require an exe wrapper for if the cross file says we don't need oneDylan Baker1-1/+1
Fixes #5970
2019-09-23mtest: TAP: ignore empty linesMarc-André Lureau1-0/+3
According to http://testanything.org/tap-specification.html "Any output line that is not a version, a plan, a test line, a diagnostic or a bail out is considered an “unknown” line. A TAP parser is required to not consider an unknown line as an error but may optionally choose to capture said line and hand it to the test harness, which may have custom behavior attached [...] TAP::Harness reports TAP syntax errors at the end of a test run". (glib gtest can generate empty lines)
2019-09-05mesonlib.split_args/quote_arg/join_argsAleksey Gurtovoy1-4/+3
2019-08-24wine: Try to get the short paths when generating WINEPATHThibault Saunier1-10/+10
The size of WINEPATH is limited (1024 [until recently]), we can very easily reach that limit, and even the new one (2048) so try to keep path as small as possible by using the shortPath version of paths. Also assert that we do not reach the new hard limit. And avoid having duplicates in the list of path. [until recently]: https://bugs.winehq.org/show_bug.cgi?id=45810
2019-07-30DOC: meson test multiple --suite specifications [skip ci]Michael Hirsch, Ph.D1-2/+4
docs:wrap < 80 col
2019-07-27remove vestigial $ from test error msgMichael Hirsch, Ph.D1-14/+13
Also: * cleanup os.path use to pathlib.Path in associated code * cleanup subprocess.Popen to subprocess.run
2019-07-12Change meson test's build failure exit code to 125Christoph Weiss1-1/+4
2019-06-21Made TestHarness a context manager to ensure files get closed.Jussi Pakkanen1-14/+25
2019-05-15mtest: annotate and fix bugs found by annotatingDylan Baker1-97/+104
This started out with a bug report of mtest trying to add bytes + str, which I though "Oh, mypy can help!" and turned into an entire day of awful code traversal and trying to figure out why attributes were changing type. Hopefully this makes everything cleaner and easier to follow.
2019-05-14mtest: precompile regular expressionsDylan Baker1-13/+13
This should make the matching a little bit faster, and is nicer to look at.
2019-05-14mtest: reorder importsDylan Baker1-16/+22
I wanted to look at the imports for annotations but was having hard time reading them because they're just all over the place. This is purely a human readability issue.
2019-05-02mtest: check for an empty suite listPeter Hutterer1-1/+1
For consistency, it can be useful to have an explicit empty test suite list for a test: test('test-name', binary, suite: []) This currently passes meson but fails when running meson tests: Traceback (most recent call last): File "/usr/lib/python3.7/site-packages/mesonbuild/mesonmain.py", line 122, in run return options.run_func(options) File "/usr/lib/python3.7/site-packages/mesonbuild/mtest.py", line 1005, in run return th.doit() File "/usr/lib/python3.7/site-packages/mesonbuild/mtest.py", line 756, in doit self.run_tests(tests) File "/usr/lib/python3.7/site-packages/mesonbuild/mtest.py", line 896, in run_tests visible_name = self.get_pretty_suite(test) File "/usr/lib/python3.7/site-packages/mesonbuild/mtest.py", line 875, in get_pretty_suite rv = TestHarness.split_suite_string(test.suite[0])[0] IndexError: list index out of range Fix it by simply checking for the test suite to be a valid list we can pass on Fixes #5340 Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
2019-04-21Report xpass results as failures.Jussi Pakkanen1-3/+5
2019-04-07mtest: fix TAP with --verbose (#5160)Paolo Bonzini1-8/+11
* mtest: fix TAP with --verbose TAP needs to process the test stdout even if --verbose is passed. Capture it to a separate temporary file, and print it at the end of the test if --verbose was passed. In the future, we could parse it on the fly and print the result of each TAP test point in verbose mode. * Prefer "stderr is stdout" to "==" The previous commit used "==" in accordance with the preexisting code, but reviewers preferred using "is" instead. Fix both occurrences.
2019-03-27do not use Executor if only one process was requestedPaolo Bonzini1-1/+1
This will make verbose output nicer, since it will not interleave the test stdout/stderr and the test results.
2019-03-04Merge pull request #2601 from 1ace/feature/completionJussi Pakkanen1-2/+2
Add completion scripts for Bash and Zsh
2019-03-02mtest: implement TAP parsingPaolo Bonzini1-1/+186
This provides an initial support for parsing TAP output. It detects failures and skipped tests without relying on exit code, as well as early termination of the test due to an error or a crash. For now, subtests are not recorded in the TestRun object. However, because the TAP output goes on stdout, it is printed by --print-errorlogs when a test does not behave as expected. Handling subtests as TestRuns, and serializing them to JSON, can be added later. The parser was written specifically for Meson, and comes with its own test suite. Fixes #2923.
2019-03-02mtest: add support for hard errorsPaolo Bonzini1-3/+11
Hard errors also come from the GNU Automake test protocol. They happen when e.g., the set-up of a test case scenario fails, or when some other unexpected or highly undesirable condition is encountered. TAP will use them for parse errors too. Add them to the exitcode protocol first.
generated by cgit v1.1 at 2024-08-30 02:46:07 +0000