From bb8b9ac0ba5382bcf02f614b5b3a84c3699cf52c Mon Sep 17 00:00:00 2001 From: "Henrik G. Olsson" Date: Tue, 1 Oct 2024 15:57:10 -0700 Subject: [Utils] Add new --update-tests flag to llvm-lit (#108425) This adds a flag to lit for detecting and updating failing tests when possible to do so automatically. The flag uses a plugin architecture where config files can add additional auto-updaters for the types of tests in the test suite. When a test fails with `--update-tests` enabled lit passes the test RUN invocation and output to each registered test updater until one of them signals that it updated the test (or all test updaters have been run). As such it is the responsibility of the test updater to only update tests where it is reasonably certain that it will actually fix the test, or come close to doing so. Initially adds support for UpdateVerifyTests and UpdateTestChecks. The flag is currently only implemented for lit's internal shell, so `--update-tests` implies `LIT_USE_INTERNAL_SHELL=1`. Builds on work in #97369 Fixes #81320 --- llvm/docs/CommandGuide/lit.rst | 5 +++++ 1 file changed, 5 insertions(+) (limited to 'llvm/docs/CommandGuide') diff --git a/llvm/docs/CommandGuide/lit.rst b/llvm/docs/CommandGuide/lit.rst index c9d5bab..dadecef 100644 --- a/llvm/docs/CommandGuide/lit.rst +++ b/llvm/docs/CommandGuide/lit.rst @@ -313,6 +313,11 @@ ADDITIONAL OPTIONS List all of the discovered tests and exit. +.. option:: --update-tests + + Pass failing tests to functions in the ``lit_config.update_tests`` list to + check whether any of them know how to update the test to make it pass. + EXIT STATUS ----------- -- cgit v1.1