aboutsummaryrefslogtreecommitdiff
path: root/data/test.schema.json
AgeCommit message (Collapse)AuthorFilesLines
2021-08-18test runner: add the ability to configure test.json for python pathsEli Schwartz1-0/+3
Adds new installed file types with @VAR@ substitution.
2021-07-07Allow `skip_*` test.json keys at top-level or in `matrix:`Jon Turney1-0/+18
2021-07-07Add `skip_on_os` to test.jsonJon Turney1-0/+6
2021-07-07Add `skip_on_jobname` to test.jsonJon Turney1-0/+6
Plan to replace the hard-coded list of 'may be skipped' framework tests in skippable() with annotations in test.json which record 'will be skipped in these specific CI jobs'. If the value of the MESON_CI_JOBNAME env var (an arbitrary string expected to be unique for each CI configuration) contains any of the strings in the `skip_on_jobname` key in test.json, the test is expected to output MESON_SKIP_TEST. Unexpected skips or runs are treated as an error. Future work: Maybe we should add additional count categories 'unexpected skip' and 'unexpected not skipped', rather than counting those as 'skipped' and 'failed', respectively.
2021-06-24Fix test.json schema for 'matrix'Jon Turney1-5/+11
I'm not sure what the current schema is describing, but it's not doing anything useful to validate 'matrix' entries, as I discover when I come to add something to that. 'matrix' is a dict with possible keys 'options' and 'exclude'. 'options' is a dict with arbitrary keys, whose's values are arrays Once we describe those dicts correctly, also fix that 'val' keys may be string, boolean or null, and the corresponding 'exclude' keys may be string or boolean. v2: Also allow 'val' and 'exclude' to be an array of strings.
2021-06-23Enforce all dicts with a fixed set of keys in test.json schemaJon Turney1-0/+9
Improve test.json schema to disallow arbitrary keys in all dicts which have a defined set of keys (the 'installed', matrix 'options' and 'stdout' dicts). Add 'count' and 'comment' keys to 'stdout' dict.
2020-08-27test: Add 'dir' support for installed files in test.jsonDaniel Mensinger1-0/+1
This is useful for automatically generated docs (doxygen, hotdoc) with a lot of generated files that may differ with different versions of the generator.
2020-04-30Update test.json schema to add stdoutJon Turney1-0/+25
Update the test.json schema, adding the 'stdout' property. Also amend the test.json schema so the presence of an unexpected property on the root object causes a validation error. v2: Also add 'tools' property to json schema. Amend the documentation not to use the word 'list' to describe a dict.
2020-04-17Add a json schema for the test.json used in testsDylan Baker1-0/+105
This does a couple of nice things, one is that editors like vscode can be configured to use this schema to provide auto completion and error highlighting if invalid values are added or required values are missing. It also allows us test that the format of the test matrix work in a unit test, which I've added. It does require that the python jsonschema package is installed.