From f867bfbce07aae6ed7a5b38c583490af3ea13af9 Mon Sep 17 00:00:00 2001 From: Jon Turney Date: Tue, 21 Jan 2020 00:37:29 +0000 Subject: Add a mechanism for validating meson output in tests Expected stdout lines must match lines from the actual stdout, in the same order. Lines with match type 're' are regex matched. v2: Ignore comment lines in expected_stdout v3: Automatically adjust path separators for location in expected output v4: Put expected stdout in test.json, rather than a separate file --- docs/markdown/Contributing.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) (limited to 'docs') diff --git a/docs/markdown/Contributing.md b/docs/markdown/Contributing.md index 8a24e0b..34cd8ca 100644 --- a/docs/markdown/Contributing.md +++ b/docs/markdown/Contributing.md @@ -334,6 +334,27 @@ If a tool is specified, it has to be present in the environment, and the version requirement must be fulfilled match. Otherwise, the entire test is skipped (including every element in the test matrix). +#### stdout + +The `stdout` key contains a list of dicts, describing the expected stdout. + +Each dict contains the following keys: + +- `line` +- `match` (optional) + +Each item in the list is matched, in order, against the remaining actual stdout +lines, after any previous matches. If the actual stdout is exhausted before +every item in the list is matched, the expected output has not been seen, and +the test has failed. + +The `match` element of the dict determines how the `line` element is matched: + +| Type | Description | +| -------- | ----------------------- | +| `literal` | Literal match (default) | +| `re` | regex match | + ### Skipping integration tests Meson uses several continuous integration testing systems that have slightly -- cgit v1.1 From 1dee6c618d879c20bf0a70eebc54bc8caf47716f Mon Sep 17 00:00:00 2001 From: Jon Turney Date: Tue, 28 Apr 2020 00:45:08 +0100 Subject: Update test.json schema to add stdout Update the test.json schema, adding the 'stdout' property. Also amend the test.json schema so the presence of an unexpected property on the root object causes a validation error. v2: Also add 'tools' property to json schema. Amend the documentation not to use the word 'list' to describe a dict. --- docs/markdown/Contributing.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) (limited to 'docs') diff --git a/docs/markdown/Contributing.md b/docs/markdown/Contributing.md index 34cd8ca..b16f615 100644 --- a/docs/markdown/Contributing.md +++ b/docs/markdown/Contributing.md @@ -329,10 +329,10 @@ Currently supported values are: #### tools -This section specifies a list of tool requirements in a simple key-value format. +This section specifies a dict of tool requirements in a simple key-value format. If a tool is specified, it has to be present in the environment, and the version -requirement must be fulfilled match. Otherwise, the entire test is skipped -(including every element in the test matrix). +requirement must be fulfilled. Otherwise, the entire test is skipped (including +every element in the test matrix). #### stdout -- cgit v1.1