aboutsummaryrefslogtreecommitdiff
path: root/llvm/utils/lit/tests/shtest-shell.py
AgeCommit message (Collapse)AuthorFilesLines
2024-08-14[llvm-lit][test][NFC] Moved cat command tests into separate lit test file ↵Connie1-17/+1
(#102366) This patch separates the lit tests that check for the functionality of lit's built-in cat command into its own test file and folder. This is a prerequisite for https://github.com/llvm/llvm-project/pull/101530.
2023-10-20[lit] Clean up internal shell parse errors with ScriptFatal (#68496)Joel E. Denny1-5/+13
Without this patch, the functions `executeScriptInternal` and thus `runOnce` in `llvm/utils/lit/lit/TestRunner.py` return either a tuple like `(out, err, exitCode, timeoutInfo)` or a `lit.Test.Result` object. They return the latter only when there's a lit internal shell parse error in a RUN line. In my opinion, a more straight-forward way to handle exceptional cases like that is to use python exceptions. For that purpose, this patch introduces `ScriptFatal`. Thus, this patch changes `executeScriptInternal` to always either return the tuple or raise the `ScriptFatal` exception. It updates `runOnce` and `libcxx/utils/libcxx/test/format.py` to catch the exception rather than check for the special return type. This patch also changes `runOnce` to convert the exception to a `Test.UNRESOLVED` result instead of `TEST.FAIL`. The former is the proper result for such a malformed test, for which a rerun (given an `ALLOW_RETRIES:`) serves no purpose. There are at least two benefits from this change. First, `_runShTest` no longer has to specially and cryptically handle this case to avoid unnecessary reruns. Second, an `XFAIL:` directive no longer hides such a failure [as we saw previously](https://reviews.llvm.org/D154987#4501125). To facilitate the `_runShTest` change, this patch inserts the internal shell parse error diagnostic into the format of the test's normal debug output rather than suppressing the latter entirely. That change is also important for [D154987](https://reviews.llvm.org/D154987), which proposes to reuse `ScriptFatal` for python compile errors in PYTHON lines or in `config.prologue`. In that case, the diagnostic might follow debugging output from the test's previous RUN or PYTHON lines, so suppressing the normal debug output would lose information.
2023-09-19[lit] Improve test output from lit's internal shellJoel E. Denny1-403/+453
This patch and D154984 were discussed in <https://discourse.llvm.org/t/rfc-improving-lits-debug-output/72839>. Motivation ---------- D154984 removes the "Script:" section that lit prints along with a test's output, and it makes -v and -a imply -vv. For example, after D154984, the "Script:" section below is never shown, but -v is enough to produce the execution trace following it: ``` Script: -- : 'RUN: at line 1'; echo hello | FileCheck bogus.txt && echo success -- Exit Code: 2 Command Output (stdout): -- $ ":" "RUN: at line 1" $ "echo" "hello" # command output: hello $ "FileCheck" "bogus.txt" # command stderr: Could not open check file 'bogus.txt': No such file or directory error: command failed with exit status: 2 -- ``` In the D154984 review, some reviewers point out that they have been using the "Script:" section for copying and pasting a test's shell commands to a terminal window. The shell commands as printed in the execution trace can be harder to copy and paste for the following reasons: - They drop redirections and break apart RUN lines at `&&`, `|`, etc. - They add `$` at the start of every command, which makes it hard to copy and paste multiple commands in bulk. - Command stdout, stderr, etc. are interleaved with the commands and are not clearly delineated. - They don't always use proper shell quoting. Instead, they blindly enclose all command-line arguments in double quotes. Changes ------- D154984 plus this patch converts the above example into: ``` Exit Code: 2 Command Output (stdout): -- # RUN: at line 1 echo hello | FileCheck bogus-file.txt && echo success # executed command: echo hello # .---command stdout------------ # | hello # `----------------------------- # executed command: FileCheck bogus-file.txt # .---command stderr------------ # | Could not open check file 'bogus-file.txt': No such file or directory # `----------------------------- # error: command failed with exit status: 2 -- ``` Thus, this patch addresses the above issues as follows: - The entire execution trace can be copied and pasted in bulk to a terminal for correct execution of the RUN lines, which are printed intact as they appeared in the original RUN lines except lit substitutions are expanded. Everything else in the execution trace appears in shell comments so it has no effect in a terminal. - Each of the RUN line's commands is repeated (in shell comments) as it executes to show (1) that the command actually executed (e.g., `echo success` above didn't) and (2) what stdout, stderr, non-zero exit status, and output files are associated with the command, if any. Shell quoting in the command is now correct and minimal but is not necessarily the original shell quoting from the RUN line. - The start and end of the contents of stdout, stderr, or an output file is now delineated clearly in the trace. To help produce some of the above output, this patch extends lit's internal shell with a built-in `@echo` command. It's like `echo` except lit suppresses the normal execution trace for `@echo` and just prints its stdout directly. For now, `@echo` isn't documented for use in lit tests. Without this patch, libcxx's custom lit test format tries to parse the stdout from `lit.TestRunner.executeScriptInternal` (which runs lit's internal shell) to extract the stdout and stderr produced by shell commands, and that parse no longer works after the above changes. This patch makes a small adjustment to `lit.TestRunner.executeScriptInternal` so libcxx can just request stdout and stderr without an execution trace. (As a minor drive-by fix that came up in testing: lit's internal `not` command now always produces a numeric exit status and never `True`.) Caveat ------ This patch only makes the above changes for lit's internal shell. In most cases, we do not know how to force external shells (e.g., bash, sh, window's `cmd`) to produce execution traces in the manner we want. To configure a test suite to use lit's internal shell (which is usually better for test portability than external shells anyway), add this to the test suite's `lit.cfg` or other configuration file: ``` config.test_format = lit.formats.ShTest(execute_external=False) ``` Reviewed By: MaskRay, awarzynski Differential Revision: https://reviews.llvm.org/D156954
2023-09-07Revert "[lit] Improve test output from lit's internal shell"Joel E. Denny1-453/+403
This reverts commit c981c533055e14302e7bff5d6898c9308065f665. The reason for the revert is discussed at: https://discourse.llvm.org/t/rfc-improving-lits-debug-output/72839/52
2023-09-07Revert "[lit] Try to fix c981c533055e test fails under windows"Joel E. Denny1-10/+10
This reverts commit f254bbf23374190c88a2b1a5f163622fbec9a936. The reason for the revert is discussed at: https://discourse.llvm.org/t/rfc-improving-lits-debug-output/72839/52
2023-09-07Revert "[lit] Fix f254bbf23374 FileCheck patterns"Joel E. Denny1-10/+10
This reverts commit 3db5db92d746bad8ba1762ca290a176e25d48565. The reason for the revert is discussed at: https://discourse.llvm.org/t/rfc-improving-lits-debug-output/72839/52
2023-09-07Revert "[lit] Fix c981c533055e's remaining test fails under windows"Joel E. Denny1-1/+1
This reverts commit 012d844fb856a89368aca95ca994726554b90f22. The reason for the revert is discussed at: https://discourse.llvm.org/t/rfc-improving-lits-debug-output/72839/52
2023-08-29[lit] Fix c981c533055e's remaining test fails under windowsJoel E. Denny1-1/+1
For `llvm/utils/lit/tests/shtest-output-printing.py`, the executable in `%{python}` wasn't properly shell-quoted for windows. This caused the 127 exit code mentioned in f254bbf23374. Fix quoting and expect exit code 1 again. Fix shell-quoting issue in a few more file names in `llvm/utils/lit/tests/shtest-shell.py`, missed in f254bbf23374. Test failures seen in <https://lab.llvm.org/buildbot/#/builders/216/builds/26436>.
2023-08-29[lit] Fix f254bbf23374 FileCheck patternsJoel E. Denny1-10/+10
Handle the case when shell quotes are not necessary.
2023-08-29[lit] Try to fix c981c533055e test fails under windowsJoel E. Denny1-10/+10
Failures were seen at <https://lab.llvm.org/buildbot/#/builders/216/builds/26431>. All but one failure is due to different shell-quoting of file names because they contain special characters under windows. Generalize associated FileCheck patterns. `llvm/utils/lit/tests/shtest-output-printing.py` fails because the exit code was 127 instead of the expected 1. Unfortunately, this CI config doesn't pass `-dump-input-filter=all` to FileCheck, so we cannot see the rest of the lit execution trace. For now, generalize the FileCheck pattern to accept any non-zero exit code to get past this error.
2023-08-29[lit] Improve test output from lit's internal shellJoel E. Denny1-403/+453
This patch and D154984 were discussed in <https://discourse.llvm.org/t/rfc-improving-lits-debug-output/72839>. Motivation ---------- D154984 removes the "Script:" section that lit prints along with a test's output, and it makes -v and -a imply -vv. For example, after D154984, the "Script:" section below is never shown, but -v is enough to produce the execution trace following it: ``` Script: -- : 'RUN: at line 1'; echo hello | FileCheck bogus.txt && echo success -- Exit Code: 2 Command Output (stdout): -- $ ":" "RUN: at line 1" $ "echo" "hello" # command output: hello $ "FileCheck" "bogus.txt" # command stderr: Could not open check file 'bogus.txt': No such file or directory error: command failed with exit status: 2 -- ``` In the D154984 review, some reviewers point out that they have been using the "Script:" section for copying and pasting a test's shell commands to a terminal window. The shell commands as printed in the execution trace can be harder to copy and paste for the following reasons: - They drop redirections and break apart RUN lines at `&&`, `|`, etc. - They add `$` at the start of every command, which makes it hard to copy and paste multiple commands in bulk. - Command stdout, stderr, etc. are interleaved with the commands and are not clearly delineated. - They don't always use proper shell quoting. Instead, they blindly enclose all command-line arguments in double quotes. Changes ------- D154984 plus this patch converts the above example into: ``` Exit Code: 2 Command Output (stdout): -- # RUN: at line 1 echo hello | FileCheck bogus-file.txt && echo success # executed command: echo hello # .---command stdout------------ # | hello # `----------------------------- # executed command: FileCheck bogus-file.txt # .---command stderr------------ # | Could not open check file 'bogus-file.txt': No such file or directory # `----------------------------- # error: command failed with exit status: 2 -- ``` Thus, this patch addresses the above issues as follows: - The entire execution trace can be copied and pasted in bulk to a terminal for correct execution of the RUN lines, which are printed intact as they appeared in the original RUN lines except lit substitutions are expanded. Everything else in the execution trace appears in shell comments so it has no effect in a terminal. - Each of the RUN line's commands is repeated (in shell comments) as it executes to show (1) that the command actually executed (e.g., `echo success` above didn't) and (2) what stdout, stderr, non-zero exit status, and output files are associated with the command, if any. Shell quoting in the command is now correct and minimal but is not necessarily the original shell quoting from the RUN line. - The start and end of the contents of stdout, stderr, or an output file is now delineated clearly in the trace. To help produce some of the above output, this patch extends lit's internal shell with a built-in `@echo` command. It's like `echo` except lit suppresses the normal execution trace for `@echo` and just prints its stdout directly. For now, `@echo` isn't documented for use in lit tests. Without this patch, libcxx's custom lit test format tries to parse the stdout from `lit.TestRunner.executeScriptInternal` (which runs lit's internal shell) to extract the stdout and stderr produced by shell commands, and that parse no longer works after the above changes. This patch makes a small adjustment to `lit.TestRunner.executeScriptInternal` so libcxx can just request stdout and stderr without an execution trace. (As a minor drive-by fix that came up in testing: lit's internal `not` command now always produces a numeric exit status and never `True`.) Caveat ------ This patch only makes the above changes for lit's internal shell. In most cases, we do not know how to force external shells (e.g., bash, sh, window's `cmd`) to produce execution traces in the manner we want. To configure a test suite to use lit's internal shell (which is usually better for test portability than external shells anyway), add this to the test suite's `lit.cfg` or other configuration file: ``` config.test_format = lit.formats.ShTest(execute_external=False) ``` Reviewed By: MaskRay, awarzynski Differential Revision: https://reviews.llvm.org/D156954
2022-09-21[lit] Implement DEFINE and REDEFINE directivesJoel E. Denny1-0/+2
These directives define per-test lit substitutions. The concept was discussed at <https://discourse.llvm.org/t/iterating-lit-run-lines/62596/10>. For example, the following directives can be inserted into a test file to define `%{cflags}` and `%{fcflags}` substitutions with empty initial values, which serve as the parameters of another newly defined `%{check}` substitution: ``` // DEFINE: %{cflags} = // DEFINE: %{fcflags} = // DEFINE: %{check} = %clang_cc1 %{cflags} -emit-llvm -o - %s | \ // DEFINE: FileCheck %{fcflags} %s ``` The following directives then redefine the parameters before each use of `%{check}`: ``` // REDEFINE: %{cflags} = -foo // REDEFINE: %{fcflags} = -check-prefix=FOO // RUN: %{check} // REDEFINE: %{cflags} = -bar // REDEFINE: %{fcflags} = -check-prefix=BAR // RUN: %{check} ``` Of course, `%{check}` would typically be more elaborate, increasing the benefit of the reuse. One issue is that the strings `DEFINE:` and `REDEFINE:` already appear in 5 tests. This patch adjusts those tests not to use those strings. Our prediction is that, in the vast majority of cases, if a test author mistakenly uses one of those strings for another purpose, the text appearing after the string will not happen to have the syntax required for these directives. Thus, the test author will discover the mistake immediately when lit reports the syntax error. This patch also expands the documentation on existing lit substitution behavior. Reviewed By: jhenderson, MaskRay, awarzynski Differential Revision: https://reviews.llvm.org/D132513
2021-08-27[llvm] [lit] Support forcing lexical test orderMichał Górny1-7/+2
Add a new --order option to choose between available test orders: the default "smart" order, predictable "lexical" order or "random" order. Default to using lexical order and one job in the lit test suite. Differential Revision: https://reviews.llvm.org/D107695
2021-03-22[NFCI][lit] Unbreak more lit self-tests after D98179Roman Lebedev1-8/+11
All of these depend on the order of tests, so if one runs them twice, the tests within them will naturally be reordered using the previous run times, which breaks them.
2021-03-16[lit] Sort test start times based on prior test timing dataDavid Zarzycki1-0/+2
Lit as it exists today has three hacks that allow users to run tests earlier: 1) An entire test suite can set the `is_early` boolean. 2) A very recently introduced "early_tests" feature. 3) The `--incremental` flag forces failing tests to run first. All of these approaches have problems. 1) The `is_early` feature was until very recently undocumented. Nevertheless it still lacks testing and is a imprecise way of optimizing test starting times. 2) The `early_tests` feature requires manual updates and doesn't scale. 3) `--incremental` is undocumented, untested, and it requires modifying the *source* file system by "touching" the file. This "touch" based approach is arguably a hack because it confuses editors (because it looks like the test was modified behind the back of the editor) and "touching" the test source file doesn't work if the test suite is read only from the perspective of `lit` (via advanced filesystem/build tricks). This patch attempts to simplify and address all of the above problems. This patch formalizes, documents, tests, and defaults lit to recording the execution time of tests and then reordering all tests during the next execution. By reordering the tests, high core count machines run faster, sometimes significantly so. This patch also always runs failing tests first, which is a positive user experience win for those that didn't know about the hidden `--incremental` flag. Finally, if users want, they can _optionally_ commit the test timing data (or a subset thereof) back to the repository to accelerate bots and first-time runs of the test suite. Reviewed By: jhenderson, yln Differential Revision: https://reviews.llvm.org/D98179
2020-07-14[lit] Prevent hang when lit sees non-ASCII charactersRichard Barton1-7/+15
As per discussion in D69207, have lit ignore UnicodeDecodeErrors when running with python 2 in an ASCII shell. Differential Revision: https://reviews.llvm.org/D82754
2020-06-05[lit] Improve naming of test result categoriesJulian Lettner1-1/+1
Improve consistency when printing test results: Previously we were using different labels for group names (the header for the list of, e.g., failing tests) and summary count lines. For example, "Failing Tests"/"Unexpected Failures". This commit changes lit to label things consistently. Improve wording of labels: When talking about individual test results, the first word in "Unexpected Failures", "Expected Passes", and "Individual Timeouts" is superfluous. Some labels contain the word "Tests" and some don't. Let's simplify the names. Before: ``` Failing Tests (1): ... Expected Passes : 3 Unexpected Failures: 1 ``` After: ``` Failed Tests (1): ... Passed: 3 Failed: 1 ``` Reviewed By: ldionne Differential Revision: https://reviews.llvm.org/D77708
2019-12-17[lit] Fix internal diff newlines for -w/-bJoel E. Denny1-1/+33
For example, without this patch: ``` $ python $LIT_BUILTINS/diff.py -b foo.txt bar.txt *** /tmp/foo.txt --- /tmp/bar.txt *************** *** 1,2 **** 1! 2--- 1,2 ---- 1! 20 ``` With this patch: ``` $ python $LIT_BUILTINS/diff.py -b foo.txt bar.txt *** /tmp/foo.txt --- /tmp/bar.txt *************** *** 1,2 **** 1 ! 2 --- 1,2 ---- 1 ! 20 ``` Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D71577
2019-10-29[lit] Extend internal diff to support `-` argumentJoel E. Denny1-1/+114
When using lit's internal shell, RUN lines like the following accidentally execute an external `diff` instead of lit's internal `diff`: ``` # RUN: program | diff file - ``` Such cases exist now, in `clang/test/Analysis` for example. We are preparing patches to ensure lit's internal `diff` is called in such cases, which will then fail because lit's internal `diff` doesn't recognize `-` as a command-line option. This patch adds support for `-` to mean stdin. Reviewed By: probinson, rnk Differential Revision: https://reviews.llvm.org/D67643
2019-10-29[lit] Make internal diff work in pipelinesJoel E. Denny1-33/+45
When using lit's internal shell, RUN lines like the following accidentally execute an external `diff` instead of lit's internal `diff`: ``` # RUN: program | diff file - # RUN: not diff file1 file2 | FileCheck %s ``` Such cases exist now, in `clang/test/Analysis` for example. We are preparing patches to ensure lit's internal `diff` is called in such cases, which will then fail because lit's internal `diff` cannot currently be used in pipelines and doesn't recognize `-` as a command-line option. To enable pipelines, this patch moves lit's `diff` implementation into an out-of-process script, similar to lit's `cat` implementation. A follow-up patch will implement `-` to mean stdin. Also, when lit's `diff` prints differences to stdout in Windows, this patch ensures it always terminate lines with `\n` not `\r\n`. That way, strict FileCheck directives checking the `diff` output succeed in both Linux and Windows. This wasn't an issue when `diff` was internal to lit because `diff` didn't then write to the true stdout, which is where the `\n` -> `\r\n` conversion happened in Python. Reviewed By: probinson, stella.stamenova Differential Revision: https://reviews.llvm.org/D66574
2019-10-25[lit] Don't fail when printing test output with special charsJoel E. Denny1-1/+13
This addresses a UnicodeEncodeError when using Python 3.6.5 in Windows 10. Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D69207
2019-10-17Revert r375114: "[lit] Make internal diff work in pipelines"Joel E. Denny1-45/+33
This series of patches still breaks a Windows bot. llvm-svn: 375121
2019-10-17Revert r375116: "[lit] Extend internal diff to support `-` argument"Joel E. Denny1-114/+1
This series of patches still breaks a Windows bot. llvm-svn: 375120
2019-10-17[lit] Extend internal diff to support `-` argumentJoel E. Denny1-1/+114
When using lit's internal shell, RUN lines like the following accidentally execute an external `diff` instead of lit's internal `diff`: ``` # RUN: program | diff file - ``` Such cases exist now, in `clang/test/Analysis` for example. We are preparing patches to ensure lit's internal `diff` is called in such cases, which will then fail because lit's internal `diff` doesn't recognize `-` as a command-line option. This patch adds support for `-` to mean stdin. Reviewed By: probinson, rnk Differential Revision: https://reviews.llvm.org/D67643 llvm-svn: 375116
2019-10-17[lit] Make internal diff work in pipelinesJoel E. Denny1-33/+45
When using lit's internal shell, RUN lines like the following accidentally execute an external `diff` instead of lit's internal `diff`: ``` # RUN: program | diff file - # RUN: not diff file1 file2 | FileCheck %s ``` Such cases exist now, in `clang/test/Analysis` for example. We are preparing patches to ensure lit's internal `diff` is called in such cases, which will then fail because lit's internal `diff` cannot currently be used in pipelines and doesn't recognize `-` as a command-line option. To enable pipelines, this patch moves lit's `diff` implementation into an out-of-process script, similar to lit's `cat` implementation. A follow-up patch will implement `-` to mean stdin. Reviewed By: probinson, stella.stamenova Differential Revision: https://reviews.llvm.org/D66574 llvm-svn: 375114
2019-10-16[lit] Fix internal diff's --strip-trailing-cr and use itJoel E. Denny1-1/+55
Using GNU diff, `--strip-trailing-cr` removes a `\r` appearing before a `\n` at the end of a line. Without this patch, lit's internal diff only removes `\r` if it appears as the last character. That seems useless. This patch fixes that. This patch also adds `--strip-trailing-cr` to some tests that fail on Windows bots when D68664 is applied. Based on what I see in the bot logs, I think the following is happening. In each test there, lit diff is comparing a file with `\r\n` line endings to a file with `\n` line endings. Without D68664, lit diff reads those files in text mode, which in Windows causes `\r\n` to be replaced with `\n`. However, with D68664, lit diff reads the files in binary mode instead and thus reports that every line is different, just as GNU diff does (at least under Ubuntu). Adding `--strip-trailing-cr` to those tests restores the previous behavior while permitting the behavior of lit diff to be more like GNU diff. Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D68839 llvm-svn: 375020
2019-10-16[lit] Clean up internal diff's encoding handlingJoel E. Denny1-1/+53
As suggested by rnk at D67643#1673043, instead of reading files multiple times until an appropriate encoding is found, read them once as binary, and then try to decode what was read. For Python >= 3.5, don't fail when attempting to decode the `diff_bytes` output in order to print it. Avoid failures for Python 2.7 used on some Windows bots by transforming diff output with `lit.util.to_string` before writing it to stdout. Finally, add some tests for encoding handling. Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D68664 llvm-svn: 375018
2019-10-14[lit] Extend internal diff to support -UJoel E. Denny1-1/+81
When using lit's internal shell, RUN lines like the following accidentally execute an external `diff` instead of lit's internal `diff`: ``` # RUN: program | diff -U1 file - ``` Such cases exist now, in `clang/test/Analysis` for example. We are preparing patches to ensure lit's internal `diff` is called in such cases, which will then fail because lit's internal `diff` doesn't recognize `-U` as a command-line option. This patch adds `-U` support. Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D68668 llvm-svn: 374814
2019-10-12Revert r374648: "Reland r374388: [lit] Make internal diff work in pipelines"Joel E. Denny1-32/+15
This series of patches still breaks a Windows bot. llvm-svn: 374683
2019-10-12Revert r374649: "Reland r374389: [lit] Clean up internal diff's encoding ↵Joel E. Denny1-53/+1
handling" This series of patches still breaks a Windows bot. llvm-svn: 374682
2019-10-12Revert r374650: "Reland r374390: [lit] Extend internal diff to support `-` ↵Joel E. Denny1-114/+1
argument" This series of patches still breaks a Windows bot. llvm-svn: 374681
2019-10-12Revert 374651: "Reland r374392: [lit] Extend internal diff to support -U"Joel E. Denny1-77/+1
This series of patches still breaks a Windows bot. llvm-svn: 374680
2019-10-12Revert r374652: "[lit] Fix internal diff's --strip-trailing-cr and use it"Joel E. Denny1-55/+2
This series of patches still breaks a Windows bot. llvm-svn: 374679
2019-10-12Revert r374653: "[lit] Fix a few oversights in r374651 that broke some bots"Joel E. Denny1-1/+1
This series of patches still breaks a Windows bot. llvm-svn: 374678
2019-10-12Revert r374657: "[lit] Try again to fix new tests that fail on Windows bots"Joel E. Denny1-6/+6
llvm-svn: 374664
2019-10-12[lit] Try again to fix new tests that fail on Windows botsJoel E. Denny1-6/+6
Based on the bot logs, when lit's internal diff runs on Windows, it looks like binary diffs must be decoded also for Python 2.7. Otherwise, writing the diff to stdout fails with: ``` UnicodeEncodeError: 'ascii' codec can't encode characters in position 7-8: ordinal not in range(128) ``` I did not need to decode using Python 2.7.15 under Ubuntu. When I do it anyway in that case, `errors="backslashreplace"` fails for me: ``` TypeError: don't know how to handle UnicodeDecodeError in error callback ``` However, `errors="ignore"` works, so this patch uses that, hoping it'll work on Windows as well. This patch leaves `errors="backslashreplace"` for Python >= 3.5 as there's no evidence yet that doesn't work and it produces more informative binary diffs. This patch also adjusts some lit tests to succeed for either error handler. This patch adjusts changes introduced by D68664. llvm-svn: 374657
2019-10-12Revert r374654: "[lit] Try to fix new tests that fail on Windows bots"Joel E. Denny1-20/+20
llvm-svn: 374656
2019-10-12[lit] Try to fix new tests that fail on Windows botsJoel E. Denny1-20/+20
llvm-svn: 374654
2019-10-12[lit] Fix a few oversights in r374651 that broke some botsJoel E. Denny1-1/+1
llvm-svn: 374653
2019-10-12[lit] Fix internal diff's --strip-trailing-cr and use itJoel E. Denny1-2/+55
Using GNU diff, `--strip-trailing-cr` removes a `\r` appearing before a `\n` at the end of a line. Without this patch, lit's internal diff only removes `\r` if it appears as the last character. That seems useless. This patch fixes that. This patch also adds `--strip-trailing-cr` to some tests that fail on Windows bots when D68664 is applied. Based on what I see in the bot logs, I think the following is happening. In each test there, lit diff is comparing a file with `\r\n` line endings to a file with `\n` line endings. Without D68664, lit diff reads those files with Python's universal newlines support activated, causing `\r` to be dropped. However, with D68664, lit diff reads the files in binary mode instead and thus reports that every line is different, just as GNU diff does (at least under Ubuntu). Adding `--strip-trailing-cr` to those tests restores the previous behavior while permitting the behavior of lit diff to be more like GNU diff. Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D68839 llvm-svn: 374652
2019-10-12Reland r374392: [lit] Extend internal diff to support -UJoel E. Denny1-1/+77
To avoid breaking some tests, D66574, D68664, D67643, and D68668 landed together. However, D68664 introduced an issue now addressed by D68839, with which these are now all relanding. Differential Revision: https://reviews.llvm.org/D68668 llvm-svn: 374651
2019-10-12Reland r374390: [lit] Extend internal diff to support `-` argumentJoel E. Denny1-1/+114
To avoid breaking some tests, D66574, D68664, D67643, and D68668 landed together. However, D68664 introduced an issue now addressed by D68839, with which these are now all relanding. Differential Revision: https://reviews.llvm.org/D67643 llvm-svn: 374650
2019-10-12Reland r374389: [lit] Clean up internal diff's encoding handlingJoel E. Denny1-1/+53
To avoid breaking some tests, D66574, D68664, D67643, and D68668 landed together. However, D68664 introduced an issue now addressed by D68839, with which these are now all relanding. Differential Revision: https://reviews.llvm.org/D68664 llvm-svn: 374649
2019-10-12Reland r374388: [lit] Make internal diff work in pipelinesJoel E. Denny1-15/+32
To avoid breaking some tests, D66574, D68664, D67643, and D68668 landed together. However, D68664 introduced an issue now addressed by D68839, with which these are now all relanding. Differential Revision: https://reviews.llvm.org/D66574 llvm-svn: 374648
2019-10-10Revert r374388: "[lit] Make internal diff work in pipelines"Joel E. Denny1-32/+15
This breaks a Windows bot. llvm-svn: 374429
2019-10-10Revert r374389: "[lit] Clean up internal diff's encoding handling"Joel E. Denny1-53/+1
This breaks a Windows bot. llvm-svn: 374427
2019-10-10Revert r374390: "[lit] Extend internal diff to support `-` argument"Joel E. Denny1-114/+1
This breaks a Windows bot. llvm-svn: 374426
2019-10-10Revert r374392: "[lit] Extend internal diff to support -U"Joel E. Denny1-77/+1
This breaks a Windows bot. llvm-svn: 374425
2019-10-10[lit] Extend internal diff to support -UJoel E. Denny1-1/+77
When using lit's internal shell, RUN lines like the following accidentally execute an external `diff` instead of lit's internal `diff`: ``` # RUN: program | diff -U1 file - ``` Such cases exist now, in `clang/test/Analysis` for example. We are preparing patches to ensure lit's internal `diff` is called in such cases, which will then fail because lit's internal `diff` doesn't recognize `-U` as a command-line option. This patch adds `-U` support. Reviewed By: rnk Differential Revision: https://reviews.llvm.org/D68668 llvm-svn: 374392
2019-10-10[lit] Extend internal diff to support `-` argumentJoel E. Denny1-1/+114
When using lit's internal shell, RUN lines like the following accidentally execute an external `diff` instead of lit's internal `diff`: ``` # RUN: program | diff file - ``` Such cases exist now, in `clang/test/Analysis` for example. We are preparing patches to ensure lit's internal `diff` is called in such cases, which will then fail because lit's internal `diff` doesn't recognize `-` as a command-line option. This patch adds support for `-` to mean stdin. Reviewed By: probinson, rnk Differential Revision: https://reviews.llvm.org/D67643 llvm-svn: 374390