aboutsummaryrefslogtreecommitdiff
path: root/run_tests.py
AgeCommit message (Collapse)AuthorFilesLines
2022-11-03python 3.11: suppress an incorrect EncodingWarningEli Schwartz1-0/+9
python 3.11 adds a warning that in 3.15, UTF-8 mode will be default. This is fantastic news, we'd love that. Less fantastic: this warning is silly, we *want* these checks to be affected. Plus, the recommended alternative API would (in addition to warning people when UTF-8 mode removed the problem) also require using a minimum python version of 3.11 (in which the warning was added) or add verbose if/else soup. The simple, and obvious, approach is to add a warnings filter to hide it.
2022-09-19compilers: directly import from subpackagesEli Schwartz1-4/+5
It turns out we don't generally need to proxy every compiler ever through the top-level package. The number of times we directly poke at one is negligible and direct imports are pretty clean.
2022-03-31unittests: move get_convincing_fake_env_and_cc to run_tests.pyMarvin Scholz1-1/+12
2022-03-01use a more sane check instead of run_custom_lintEli Schwartz1-0/+8
Unfortunately, checking for strings without context is exceedingly prone to false positives, while missing anything that indirectly opens a file. Python 3.10 has a feature to warn about this though -- and it uses a runtime check which runs at the same time that the code fails to open files in the broken Windows locale. Set this up automatically when running the testsuite. Sadly, Python's builtin feature to change the warning level, e.g. by setting EncodingWarning to error at startup, is utterly broken if you want to limit it to only certain modules. This is tracked in order to be more efficiently ignored at https://bugs.python.org/issue34624 and https://github.com/python/cpython/pull/9358 It is also very trigger happy and passing stuff around via environment variable either messes with the testsuite, or with thirdparty programs which are implemented in python *such as lots of gnome*, or perhaps both. Instead, add runtime code to meson itself, to add a hidden "feature". In the application source code, running the 'warnings' module, you can actually get the expected behavior that $PYTHONWARNINGS doesn't have. So check for a magic testsuite variable every time meson starts up, and if it does, then go ahead and initialize a warnings filter that makes EncodingWarning fatal, but *only* when triggered via Meson and not arbitrary subprocess scripts.
2022-02-17unittests: make ninja detection faster and more robustEli Schwartz1-8/+12
Instead of blindly assuming when $CI is set that `ninja` is always correct and always new enough, check for it the same way we do when $CI is not set. Instead of special casing when $CI is set and skipping ninja detection in subprocess tests (by assuming it is always `ninja`), skip detection in subprocess tests all the time, by passing this information around across processes after the first time it is looked up. This means that local (non-CI) tests are faster too! Fixes running unittests in alpine linux using samu. Although ninja is a symlink to samu, the exact output string on no-op builds is different, which we already handle, but what we don't handle is the fact that samu prints a third case when you invoke it as `ninja`. If $CI is exported, then the unittests ignored $NINJA entirely. Fixes running unittests when $CI is set, `samu` exists and `ninja` does not exist. This is not currently the case anywhere, but may be in the future; why not fix it now?
2022-01-16Add a test for the --vsenv meson setup optionNirbheek Chauhan1-4/+3
The tests and the unittests both unconditionally call setup_vsenv() because all tests are run using the backend commands directly: ninja, msbuild, etc. There's no way to undo this vs env setup, so the only way to test that --vsenv works is by: 1. Removing all paths in PATH that provide ninja 2. Changing setup_vsenv(force=True) to forcibly set-up a new vsenv when MESON_FORCE_VSENV_FOR_UNITTEST is set 3. Mock-patching build_command, test_command, install_command to use `meson` instead of `ninja` 4. Asserting that 'Activating VS' is in the output for all commands 5. Ensure that compilation works because ninja is picked up from the vs env. I manually checked that this test actually does fail when the previous commit is reverted.
2021-10-10Add --vsenv command line option and active VS only when neededXavier Claessens1-2/+2
2021-06-29pathlib: Patch pathlib to work around some bugs (fixes #7295)Daniel Mensinger1-1/+5
2021-06-22coverage: Enable coverage reportsDaniel Mensinger1-49/+29
2021-06-18tests: clear mesonlib.project_meson_versions (which **really** needs a refactor)Daniel Mensinger1-0/+2
2021-06-18tests: Always enable the traceback in run_project_tests.pyDaniel Mensinger1-3/+11
2021-06-09typing: Fully annotate run_project_tests.pyDaniel Mensinger1-9/+9
2021-05-29Remove HFS workaround as everyone should use APFS by now.Jussi Pakkanen1-7/+0
2021-05-13Set up VS environment automatically when run.Jussi Pakkanen1-1/+2
2021-05-08Set unittest backend with an argument rather than an envvar.Jussi Pakkanen1-2/+1
2021-04-25Xcode: add objective C flags to plain C because Xcode requires it.Jussi Pakkanen1-3/+1
2021-03-04mass rewrite of string formatting to use f-strings everywhereEli Schwartz1-8/+8
performed by running "pyupgrade --py36-plus" and committing the results
2021-03-04various python neatness cleanupsEli Schwartz1-1/+1
All changes were created by running "pyupgrade --py3-only --keep-percent-format" and committing the results. I have not touched string formatting for now. - use set literals - simplify .format() parameter naming - remove __future__ - remove default "r" mode for open() - use OSError rather than compatibility aliases - remove stray parentheses in function(generator) scopes
2021-02-23add a script to run a single testDylan Baker1-1/+2
As a Meson developer it's often frustrating to have a single functional test with a regression. These tests can be awkward to reproduce, especially when they make use of a test.json file. This script provides a simmple interface to call functional tests 1 at a time, regardless of whether they use a test.json or not. If they do use a test.json, and have a matrix, then the `--subtest` option can be used to select spcific combinations, for example: ```sh ./run_single_test.py "test cases/frameworks/15 llvm" --subtest 1 ``` will run only the second (zero indexed of course) subtest from the llvm test cases. This is not a super elegent script, but this is super useful.
2021-02-23run_*_tests: Add some type annotationsDylan Baker1-12/+19
2021-01-21mlog: add __str__ method to AnsiDecoratorPaolo Bonzini1-3/+3
Automatically colorize the text when printing the AnsiDecorator, based on the result of mlog.colorize_console(). This is how AnsiDecorator is used most of the time anyway.
2021-01-04Use a single coredata dictionary for optionsDylan Baker1-1/+1
This patches takes the options work to it's logical conclusion: A single flat dictionary of OptionKey: UserOptions. This allows us to simplify a large number of cases, as we don't need to check if an option is in this dict or that one (or any of 5 or 6, actually).
2021-01-04move OptionKey to mesonlibDylan Baker1-1/+2
There's starting to be a lot of things including coredata that coredata needs to itself include. putting it in mesonlib makes more sense
2021-01-04use OptionKey for compiler_optionsDylan Baker1-2/+2
2020-11-20use real pathlib moduleDylan Baker1-1/+1
We added the _pathlib module to work around defeciencies in python 3.5's implementation, since we now rely on 3.6 lets drop this
2020-10-04pathlib: Fix resolve() by overriding it in Python 3.5Daniel Mensinger1-1/+1
2020-09-24compilers/mixins/clike: fix mypy issuesDylan Baker1-2/+0
2020-09-17tests: Make monkey patching smarterDylan Baker1-25/+13
There are two problems here. The first is that we're doing manual monkey patching which is fragile and verbose, when unittest.mock is designed specifically to solve this exact problem. The second is that we're monkey patching os.environ at two different levels with the same information. So let's only do it once.
2020-09-10Add a test run in an environment which only has a cross compilerJon Turney1-2/+4
Add '--cross-only' option to run_tests.py, so we can arrange not to run tests in the 'native' suite when only a cross-compiler is available, as they can't succeed.
2020-09-08typing: fixup of #7688Daniel Mensinger1-2/+2
This bug not beeing caught in review is another reason for adding strict typing support to meson.
2020-06-15run_*tests*.py: print Meson version at start of test suiteMichael Hirsch1-2/+3
on some systems, tests may take over an hour to run--only to find you might have used an unintended Meson version (e.g. release instead of dev). This change prints the Meson version at the start of the run_*tests*.py scripts. Also, raise SystemExit(main()) is preferred in general over sys.exit(main())
2020-04-30Make colourize_console() a functionJon Turney1-3/+3
Currently, colourize_console is a constant, set at process initialization. To allow the actual stdout to be easily compared with the expected when running tests, we want to allow colourization to be on for the test driver, but not for the in-process configure done by run_configure, which has stdout redirected from a tty to a pipe. v2: Cache _colorize_console per file object v3: Reset cache on setup_console()
2020-04-20Compiler options per langJohn Ericson1-1/+1
A current rather untyped storage of options is one of the things that contributes to the options code being so complex. This takes a small step in synching down by storing the compiler options in dicts per language. Future work might be replacing the langauge strings with an enum, and defaultdict with a custom struct, just like `PerMachine` and `MachineChoice`.
2020-02-22ninjabackend: List PDBs in output list for targetsNirbheek Chauhan1-10/+8
This is more correct, and forces the target(s) to be rebuilt if the PDB files are missing. Increases the minimum required Ninja to 1.7, which is available in Ubuntu 16.04 under backports. We can't do the same for import libraries, because it is impossible for us to know at configure time whether or not an import library will be generated for a given DLL.
2020-01-28Specify crossfile use in CI job configurationJon Turney1-18/+14
Pull the crossfile specification out of run_test.py so it can be specified in the CI job configuration. Also make some fixes to output ordering in run_test.py.
2020-01-15run_tests: Fix detection of ninja 1.9Nirbheek Chauhan1-16/+22
`get_backend_commands()` doesn't get called when we run tests as subprocesses, so detect ninja on import. This should speed up CI. Fixes https://github.com/mesonbuild/meson/issues/5888
2019-12-13run_tests.py: use mlog.log_once to avoid spamming ninja < 1.9 warningsDylan Baker1-3/+3
2019-12-05lgtm: Fix Module imported with `import` and `import from`Daniel Mensinger1-9/+9
2019-10-20Fix all flake8 warningsDaniel Mensinger1-1/+0
2019-09-05mesonlib.split_args/quote_arg/join_argsAleksey Gurtovoy1-1/+1
2019-09-01tests: Require ninja 1.9 on CINirbheek Chauhan1-9/+19
Also, print messages when we have to enable the timestamp resolution workaround.
2019-09-01tests: Upgrade to ninja v1.9 for LinuxNirbheek Chauhan1-1/+1
Also use the upstream release. This eliminates Meson CI's dependence on my server.
2019-09-01tests: Detect ninja v1.9 and disable timestamp hackNirbheek Chauhan1-5/+18
The high-res timestamp PR by QuLogic was merged in v1.9, so we can switch back to upstream ninja.
2019-08-24Intel-Cl tests: increase robustness of Windows Intel compilers detection (#5828)Michael Hirsch, Ph.D1-3/+27
* intel-cl tests: more rigorous detection of intent to use Intel Windows compilers * fortran coarray test: make skipping more robust in that underlying MPI stack is .run() This is useful for any Fortran coarray work, and especially for intel-cl where multiple Intel compiler versions are often installed, and the wrong underlying MPI library may be dynamically linked, and so a runtime check is needed to exercise the MPI stack underlying Fortran coarray. This is done by fc.run('sync all; end', dependencies: coarray) * pep8
2019-08-12Put native file before cross file in options listJohn Ericson1-1/+1
I think this makes more sense by matching how build comes before host. Native builds are also more common than cross builds.
2019-07-30fortran tests: non-gfortran compilers on Windows are best static libsMichael Hirsch, Ph.D1-1/+1
collapsed get_relative_files_list_from_dir to a one-liner via pathlib.Path
2019-07-30run_tests: ifort / intel-cl must have ninja backendMichael Hirsch, Ph.D1-2/+4
2019-07-23cmake: added test case for environment variablesDaniel Mensinger1-6/+11
2019-07-09Add command line argument to skip unit tests.Jussi Pakkanen1-6/+12
2019-05-16Make `PerMachine` and `MachineChoice` have just `build` and `host`John Ericson1-1/+1
Meson itself *almost* only cares about the build and host platforms. The exception is it takes a `target_machine` in the cross file and exposes it to the user; but it doesn't do anything else with it. It's therefore overkill to put target in `PerMachine` and `MachineChoice`. Instead, we make a `PerThreeMachine` only for the machine infos. Additionally fix a few other things that were bugging me in the process: - Get rid of `MachineInfos` class. Since `envconfig.py` was created, it has no methods that couldn't just got on `PerMachine` - Make `default_missing` and `miss_defaulting` work functionally. That means we can just locally bind rather than bind as class vars the "unfrozen" configuration. This helps prevent bugs where one forgets to freeze a configuration.