aboutsummaryrefslogtreecommitdiff
path: root/run_project_tests.py
AgeCommit message (Collapse)AuthorFilesLines
2021-07-25Remove workaround now that we have a new Ninja.Jussi Pakkanen1-7/+0
2021-07-13Make it an error if MESON_CI_JOBNAME isn't set under CIJon Turney1-6/+3
Also check skip is as expected for CI all runs, not just those that set MESON_CI_JOBNAME.
2021-07-07Use None as Environment object build_dir in detect_system_compiler()Jon Turney1-24/+23
The Environment object constructor accepts None as build_dir (for quite a while now), so don't bother with creating a temporary directory for use as the build_dir, if we're not going to need it. Future work: Environment.__init__() sets scratch_dir to '' if build_dir is None, which seems a little wonky, as it isn't a path.
2021-07-07Annotate framework tests with where they are expected to skipJon Turney1-56/+1
Remove hard-coded framework test skip logic in skippable(), instead annotate test.json with environments in which skip is expected. (Mainly this is done with by testing the value of MESON_CI_JOBNAME now set for linux jobs)
2021-07-07Allow `skip_*` test.json keys at top-level or in `matrix:`Jon Turney1-31/+41
2021-07-07Add `skip_on_os` to test.jsonJon Turney1-0/+11
2021-07-07Add `skip_on_jobname` to test.jsonJon Turney1-4/+55
Plan to replace the hard-coded list of 'may be skipped' framework tests in skippable() with annotations in test.json which record 'will be skipped in these specific CI jobs'. If the value of the MESON_CI_JOBNAME env var (an arbitrary string expected to be unique for each CI configuration) contains any of the strings in the `skip_on_jobname` key in test.json, the test is expected to output MESON_SKIP_TEST. Unexpected skips or runs are treated as an error. Future work: Maybe we should add additional count categories 'unexpected skip' and 'unexpected not skipped', rather than counting those as 'skipped' and 'failed', respectively.
2021-07-07Drop non-framework test names from skippable()Jon Turney1-12/+0
In all these cases, the test has already been determined as skippable because it's not part of the 'frameworks' suite.
2021-07-05pyupgradeEli Schwartz1-1/+1
2021-06-29fix: Always explicitly set encoding for text files (fixes #8263)Daniel Mensinger1-11/+10
2021-06-29pathlib: Patch pathlib to work around some bugs (fixes #7295)Daniel Mensinger1-1/+5
2021-06-25Split compiler detection from EnvironmentDaniel Mensinger1-4/+5
This moves all the compiler detection logic into the new compilers.detect module. This dramatically reduces the size and complexity of Environment.
2021-06-22Fix project tests category name used in skippable() and XML outputJon Turney1-3/+4
Since 25df6e7d split the iteration over tests to start them from the iteration to collect their results, the variable 'name' is only being set in the first iteratiorn, so all tests are treated as being in the last test category read (probably 'wasm') for skipppable() and in the XML output. Store the category name in the TestDef object Use it in skippable() Use it in classname attribute of XML test results
2021-06-18tests: Always enable the traceback in run_project_tests.pyDaniel Mensinger1-1/+4
2021-06-18tests: Add visual seperator to the test logs in run_project_tests.pyDaniel Mensinger1-0/+9
2021-06-11CI: Split out file format test as a separate GitHub workflowJon Turney1-48/+0
Split out checking of file format as a separate GitHub workflow, rather than running it as part of the project tests for every platform and toolchain combination in CI, so that this test is not effected by the changed paths constraints which are applied to the project tests.
2021-06-09tests: Force colorize CI outputDaniel Mensinger1-0/+12
2021-06-09tests: Don't even start running skipped testsDaniel Mensinger1-7/+9
2021-06-09tests: Fix global state issueDaniel Mensinger1-5/+27
2021-06-09tests: Add option to limit the number of workersDaniel Mensinger1-18/+29
Also, increase the minimum ammount of workers from 1 to 2 when the core count can not be determined.
2021-06-09tests: Optional progress bar with tqdmDaniel Mensinger1-13/+36
2021-06-09tests: Refactored test runnerDaniel Mensinger1-81/+154
2021-06-09tests: Reduce complexity of the run_tests signatureDaniel Mensinger1-16/+15
2021-06-09tests: reduce global variablesDaniel Mensinger1-26/+24
2021-06-09typing: Fully annotate run_project_tests.pyDaniel Mensinger1-95/+134
2021-06-07cython: add 1 basic testDylan Baker1-2/+3
2021-05-13Set up VS environment automatically when run.Jussi Pakkanen1-2/+3
2021-05-02Add validation for --only in `run_project_tests.py`Ralf Gommers1-1/+3
This now gives a clear error rather than silently passes for unrecognized categories, like: python run_project_tests.py --only nonexisting or python run_project_tests.py --only objc # should be 'objective c'
2021-03-24Merge pull request #8568 from dcbaker/submit/qt-dependency-factoryJussi Pakkanen1-9/+12
QT: use a proper dependency factory
2021-03-23run_project_tests: add annotationsDylan Baker1-1/+1
2021-03-23run_project_tests.py: Allow qt tests to skip on !macOSDylan Baker1-8/+11
We don't always have qmake installed (and it's good to test failure paths too!) so we can't expect this to succeed in all cases. With the following commit we'll use a test.json to test both pkg-config and qmake, so we need to be able to skip.
2021-03-23test.json: Add support for not matching stdout linesXavier Claessens1-20/+53
By default expected line must be matched in order. When an expected line is matched it does not matter if it's matched again later or not. When defining "count", it means that line must be matched exactly that many times before matching the next expected line. Once all occurences have been matched for an expected line, it not must appear any more in all next lines.
2021-03-23run_project_tests.py: Allow "--only common/240" syntaxXavier Claessens1-13/+24
2021-03-16Update VS module version check.Jussi Pakkanen1-0/+2
2021-03-16Delete leftover files from some tests.Jussi Pakkanen1-0/+11
2021-03-04mass rewrite of string formatting to use f-strings everywhereEli Schwartz1-25/+25
performed by running "pyupgrade --py36-plus" and committing the results
2021-03-04various python neatness cleanupsEli Schwartz1-1/+1
All changes were created by running "pyupgrade --py3-only --keep-percent-format" and committing the results. I have not touched string formatting for now. - use set literals - simplify .format() parameter naming - remove __future__ - remove default "r" mode for open() - use OSError rather than compatibility aliases - remove stray parentheses in function(generator) scopes
2021-02-23run_*_tests: Add some type annotationsDylan Baker1-3/+8
2021-02-23run_project_tests: Delete a build directory if it already existsDylan Baker1-0/+2
This can happen if you abort a run before the cleanup finishes
2021-02-23run_project_tests: split json parsing code from test gathering codeDylan Baker1-113/+123
2021-02-06ci: Add bindgen to CI imagesDylan Baker1-0/+4
2021-01-21mlog: add __str__ method to AnsiDecoratorPaolo Bonzini1-16/+1
Automatically colorize the text when printing the AnsiDecorator, based on the result of mlog.colorize_console(). This is how AnsiDecorator is used most of the time anyway.
2021-01-10cmake: add PATH logic to preliminary dep check (fixes #8133)Daniel Mensinger1-0/+1
2020-11-23Remove AutoDeletedDirChristoph Reiter1-16/+4
Replace the only usage with a simple try/finally and remove the class.
2020-11-23Replace various calls to AutoDeletedDir with TemporaryDirectoryWinProofChristoph Reiter1-6/+6
AutoDeletedDir was a workaround to TemporaryDirectory not using windows_proof_rmtree(). TemporaryDirectoryWinProof does now, so replace it.
2020-11-22Merge pull request #8011 from dcbaker/submit/post-python36-cleanupsJussi Pakkanen1-1/+1
Python 3.6 cleanups
2020-11-21run_project_tests: Use the test environment for install and cleanJason Ekstrand1-2/+2
2020-11-20use real pathlib moduleDylan Baker1-1/+1
We added the _pathlib module to work around defeciencies in python 3.5's implementation, since we now rely on 3.6 lets drop this
2020-11-13gnome: Handle libraries that are not in the current build dirSam Thursfield1-0/+4
The generate_gir() function previously assumed all library inputs were in the current build dir. This would fail if they weren't.
2020-10-30Bump minimum supported Python version to 3.6. Closes #6297.Jussi Pakkanen1-3/+2