aboutsummaryrefslogtreecommitdiff
path: root/run_project_tests.py
AgeCommit message (Collapse)AuthorFilesLines
11 daysWhen configuring fails in Github Actions, print folded logsEli Schwartz1-8/+3
A common, and challenging, issue in CI runners is debugging issues when you know the information you want to check, but it's in the log file which you don't have because remote CI machines. There are various edge cases where this is especially hard to solve, such as inside of `pip install` where the build directory with the log file is automatically cleaned up. But it's never really *easy* when you don't expect it, and the best case scenario is your iteration time gets cut in half as you hurriedly go add some `cat`s to your CI scripts. Meson can, at least sometimes, detect platforms where text can be emitted inside of "folds", which are auto-collapsed and don't obscure the general output, but when clicked will expand the logfile contents. Hook this up. We start off with a Github Actions implementation. We had some internal code used by our own project tests runner, which can be utilized. Also permit forcing it via an environment variable, in case autodetection fails and you just want to force *something*, especially when meson is called a couple layers deep inside some other tool.
2024-07-28add initial support for llvm-flangH. Vetinari1-0/+1
2024-07-12tests: rename poorly named function and add report argumentDylan Baker1-2/+7
The poorly named `print_tool_versions()` doesn't just print the tools versions, it finds them and populates a global table, without which some tests will fail. Rename the function and add a `report` argument so that calls can decide whether they want to have the printed message, because the single runner doesn't in quick mode.
2024-07-10tests: Clean up leftover files from source dirs.Jussi Pakkanen1-1/+14
2024-05-23Refactor option classes to their own file.Jussi Pakkanen1-1/+2
2024-04-14Fix builds with Ninja 12 and remove a 5 year old workaround.Jussi Pakkanen1-2/+1
2024-04-10Do not even submit test suites that are skipped completely.Jussi Pakkanen1-0/+1
2024-04-10Add a verbose output option to run_project_tests and default it to off.Jussi Pakkanen1-2/+8
2024-04-08meson format commandCharles Brunet1-1/+3
2024-03-17tests: rename skip_on_jobname to expect_skip_on_jobname and skip_on_os to ↵Christoph Reiter1-4/+4
expect_skip_on_os The test.json format currently has three keys related to skipping tests: * `skip_on_jobname` * `skip_on_os` * `skip_on_env` While `skip_on_env` marks the test itself as skipped, i.e. they don't get run when the conditions are met, the other two skip options are just marking the test as "expected to be skipped" if the conditions apply, i.e. they want to see `MESON_SKIP_TEST` in the output and things will fail if that doesn't happen. They don't actually skip the tests as the names imply. To make this clearer rename the keys: * `skip_on_jobname` -> `expect_skip_on_jobname` * `skip_on_os` -> `expect_skip_on_os` `skip_on_env` stays the same, since that actually skips. The docs were also confused about this, so adjust that too.
2024-03-10fix reconfigure subproject base optionsCharles Brunet1-1/+1
2024-03-01Fix base and compiler options not reconfigurable.Charles Brunet1-1/+1
Fixes #12920.
2024-02-14run_project_tests: Fix Cython compiler detectionAndrew McNulty1-20/+13
On Debian systems the cython compiler binary is installed as `cython3`. The current logic for detecting whether to run the Cython unit tests instead checks for `cython` or the value of the `CYTHON` environment variable, which leads to cases where the underlying Meson can correctly compile Cython code but the test harness excludes these tests from execution because it cannot find `cython3`. This commit makes the test harness use the same detection method as Meson itself. It also takes the opportunity to refactor some existing code which does the same job for Objective C and Objective C++ tests to avoid repetition.
2023-12-24run_project_tests: fix assertion that crashes if you have git worktreesEli Schwartz1-1/+1
git worktrees are frequently a handy tool to use. This glob pattern asserts an error if the same named directory appears more than once, which it will -- once per worktree -- but only because it is globbing the entire project root, even though it should always only exclusively care about files in `test cases/`. So just glob correctly, and then I can run the testsuite locally again.
2023-12-24project tests: handle code internally fetching the meson commandEli Schwartz1-0/+1
Fixes crash when running the tests and yasm is installed, but nasm is not. When printing the compilers at early startup in run_project_tests.py, we try to detect the nasm compiler, which in this case turns out to be yasm, and its exelist assumes that the meson command is set. Fixes #12662
2023-12-22tests: skip symlink creation when not supportedCharles Brunet1-2/+5
2023-12-13Use SPDX-License-Identifier consistentlyDylan Baker1-12/+1
This replaces all of the Apache blurbs at the start of each file with an `# SPDX-License-Identifier: Apache-2.0` string. It also fixes existing uses to be consistent in capitalization, and to be placed above any copyright notices. This removes nearly 3000 lines of boilerplate from the project (only python files), which no developer cares to look at. SPDX is in common use, particularly in the Linux kernel, and is the recommended format for Meson's own `project(license: )` field
2023-12-04run_project_tests: remove debug printDylan Baker1-1/+0
2023-11-07Prohibit symlinks in test data dir because they get mangled by setup.py.Jussi Pakkanen1-0/+21
2023-10-04Remove type comments in run_project_tests.pyTristan Partin1-12/+12
2023-09-13run_project_tests: support checking for symlinksArsen Arsenović1-1/+6
2023-09-07Fix include_directories test for relative pathCharles Brunet1-0/+2
- On Windows, it was not detected if include directory was an absolute path to source directory, because of the mis of path separators. - In the edgecase the include directory begins with the exact same string as the source directory, but is a different directory, it was falsely reported as an error. Fixes #12217.
2023-09-07Fix crash when installing a vala library and python sourcesXavier Claessens1-1/+4
Installing python sources causes the python module to call create_install_data() before Ninja backends adds extra outputs to Vala targets. Target objects are supposed to be immutable, adding outputs that late is totally wrong. Add extra vala outputs immediately, but be careful because the main output is only added later in post_init(). Luckily the base class already puts a placeholder item in self.outputs for the main filename so we can just replace self.outputs[0] instead of replacing the whole list which would contain vala outputs at that stage. This is surprisingly what SharedLibrary was already doing.
2023-09-01tests: repeat the failure reason in the detailed logsEli Schwartz1-0/+3
2023-09-01tests: move a couple log files into collapsed github groups where possibleEli Schwartz1-8/+7
2023-09-01tests: move the newline to the right place in ci includesEli Schwartz1-3/+3
Make sure this is generic enough to be properly usable
2023-09-01tests: sync ninja stdout/stderr instead of listing them separatelyEli Schwartz1-2/+2
It is fairly confusing for all the "ninja explain" to be the last thing you see, instead of the compile error.
2023-09-01tests: make summary section for project tests more like pytestEli Schwartz1-5/+6
Logs for failing tests come first, then the list of pass/fail.
2023-09-01tests: make run_project_tests finish off the verbose logs with a summaryEli Schwartz1-0/+5
List each testcase name that failed so we can see at a glance which ones they are.
2023-08-23machine file: Add @GLOBAL_SOURCE_ROOT@ and @DIRNAME@Xavier Claessens1-4/+4
2023-08-17tests: consolidate MESON_SKIP_TEST reporting and use it in unittestsEli Schwartz1-19/+7
Previously, we only reported the skip reason when running project tests.
2023-08-14Python: Add 'limited_api' kwarg to extension_moduleAndrew McNulty1-2/+12
This commit adds a new keyword arg to extension_module() that enables a user to target the Python Limited API, declaring the version of the limited API that they wish to target. Two new unittests have been added to test this functionality.
2023-06-26dependencies: defer importing a custom dependency until it is usedEli Schwartz1-1/+1
This lessens the amount of code imported at Meson startup by mapping each dependency to a dictionary entry and using a programmable import to dynamically return it. Minus 16 files and 6399 lines of code imported at startup.
2023-04-24Initial support for Metrowerks C/C++ compilerNomura1-1/+3
2023-04-11fix various spelling issuesJosh Soref1-1/+1
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2023-04-04tests: make it easier to read overall results in githubEli Schwartz1-1/+7
Use collapsible sections for cicmds.
2023-03-20project tests: add log of the setup command being runEli Schwartz1-3/+6
Followup to commit 2acb6ee79e24db047800da30f5f4ac52cb9c012d.
2023-02-22remove pointless, unused typing-only import in the test runnerEli Schwartz1-1/+0
In the commit that originally added this import, it wasn't even used. Now the implementation is being moved, so it will fail to work. I do not know why I originally added it, but it needs to go. :)
2023-02-13tests: do not run the deprecated python3 tests unless python is python3Eli Schwartz1-2/+2
It's broken in e.g. pypy, but that's okay because the module is utterly deprecated, and people should use the python module instead.
2023-01-12tests: track when an entire category is skipped and be quieterEli Schwartz1-8/+10
There is no need to state for every single test that "preconditions were not met". And logging the skip reason for a single test is easy to read, but making every second line alternate is less so.
2023-01-12tests: properly log the reason why a test got unexpectedly skippedEli Schwartz1-1/+2
2022-12-15Enable Java project tests on DarwimTristan Partin1-1/+1
2022-11-23python: Use correct extension filename suffix on Python < 3.8.7Daniele Nicolodi1-4/+10
On Windows, in Python version prior to 3.8.7, the `sysconfig` modules provides an extension filename suffix that disagrees the one returned by `distutils.sysconfig`. Get the more awesome suffix from the latter when building for a Python version known to present this issue. Simplify the extension module filename suffix lookup to use the same method used by `setuptools`. Adjust project tests accordingly. Fixes #10960.
2022-11-22test runner: fix excluding null optionsMarvin Scholz1-12/+12
It was impossible to specify null arguments in the exclude array, while they were passed properly in the code, they were stringified while the entries in the options array were just completely removed. To fix this, do not early stringify the argument name and option together as that would make it impossible to properly match it to None without introducing a new special case for the string "None". So instead they are now kept as tuples and only stringified when its actually needed.
2022-11-17fix deprecated use of meson builddir/ in testsuiteEli Schwartz1-1/+1
2022-10-24basic support for oneapi compilersRobert Cohn1-1/+2
2022-08-17run_project_tests: also change paths for DEPRECATION messagesDylan Baker1-1/+1
Not just WARNING and ERROR
2022-08-16deprecate running "meson builddir" without setup subcommandEli Schwartz1-1/+1
This is ambiguous, if the build directory has the same name as a subcommand then we end up running the subcommand. It also means we have a hard time adding *new* subcommands, because if it is a popular name of a build directory then suddenly scripts that try to set up a build directory end up running a subcommand instead. The fact that we support this at all is a legacy design. Back in the day, the "meson" program was for setting up a build directory and all other tools were their own entry points, e.g. `mesontest` or `mesonconf`. Then in commit fa278f351fe3d6924b4d1961f77b5b4a36e133f8 we migrated to the subcommand mechanism. So, for backwards compatibility, we made those tools print a warning and then invoke `meson <tool>`. We also made the `meson` tool default to setup. However, we only warned for the other tools whose entry points were eventually deleted. We never warned for setup itself, we just continued to silently default to setup if no tool was provided. `meson setup` has worked since 0.42, which is 5 years old this week. It's available essentially everywhere. No one needs to use the old backwards-compatible invocation method, but it continues to drag down our ability to innovate. Let's finally do what we should have done a long time ago, and sunset it.
2022-07-06run_project_tests: Don't try to use tqdm when stdout is not a ttyDylan Baker1-1/+1
Like when piped to something like less, where tqdm just makes a mess of the output instead of providing something helpful.
2022-05-10Require CMake 3.14 for all tests that use the cmake moduleSimon McVittie1-1/+5
Older versions are not supported by the cmake module since 0.62. This avoids having to hard-code the linux-bionic-gcc CI job as being unable to run these tests, which leaves other older environments like Debian 10 still trying to run them (and failing). Signed-off-by: Simon McVittie <smcv@collabora.com>