aboutsummaryrefslogtreecommitdiff
path: root/lldb/packages/Python/lldbsuite/test/configuration.py
AgeCommit message (Collapse)AuthorFilesLines
2025-09-04[lldb] Add utility to create Mach-O corefile from YAML desc (#153911)Jason Molenda1-0/+11
I've wanted a utility to create a corefile for test purposes given a bit of memory and regsters, for a while. I've written a few API tests over the years that needed exactly this capability -- we have several one-off Mach-O corefile creator utility in the API testsuite to do this. But it's a lot of boilerplate when you only want to specify some register contents and memory contents, to create an API test. This adds yaml2mach-core, a tool that should build on any system, takes a yaml description of register values for one or more threads, optionally memory values for one or more memory regions, and can take a list of UUIDs that will be added as LC_NOTE "load binary" metadata to the corefile so binaries can be loaded into virtual address space in a test scenario. The format of the yaml file looks like ``` cpu: armv7m # optionally specify the number of bits used for addressing # (this line is from a different, 64-bit, yaml file) addressable-bits: num-bits: 39 # optionally specify one or more binary UUID and slide/virtual address to be added as an LC_NOTE # (this line is from a different, 64-bit, yaml file) binaries: - name: debug-binary.development uuid: 67942352-5857-3D3D-90CB-A3F80BA67B04 virtual-address: 0xfffffff01840c000 threads: - regsets: - flavor: gpr registers: [{name: sp, value: 0x2000fe70}, {name: r7, value: 0x2000fe80}, {name: pc, value: 0x0020392c}, {name: lr, value: 0x0020392d}] memory-regions: # stack memory - addr: 0x2000fe70 UInt32: [ 0x0000002a, 0x20010e58, 0x00203923, 0x00000001, 0x2000fe88, 0x00203911, 0x2000ffdc, 0xfffffff9 ] # instructions of a function - addr: 0x203910 UInt8: [ 0xf8, 0xb5, 0x04, 0xaf, 0x06, 0x4c, 0x07, 0x49, 0x74, 0xf0, 0x2e, 0xf8, 0x01, 0xac, 0x74, 0xf0 ] ``` and that's all that is needed to specify a corefile where four register values are specified (the others will be set to 0), and two memory regions will be emitted. The memory can be specified as an array of UInt8, UInt32, or UInt64, I anticipate that some of these corefiles may have stack values constructed manually and it may be simpler for a human to write them in a particular grouping of values. I needed this utility for an upcoming patch for ARM Cortex-M processors, to create a test for the change. I took the opportunity to remove two of the "trivial mach-o corefile" creator utilities I've written in the past, which also restricted the tests to only run on Darwin systems because I was using the system headers for Mach-O constant values. rdar://110663219
2025-07-31[lldb] Support Darwin cross compilation for remote Linux test suite runs ↵Jonas Devlieghere1-0/+4
(#151403) Fix cross-compilation of test inferiors on Darwin, targeting remote Linux. This requires specifying the target triple and using LLD for linking. Fixes #150806
2025-05-23[lldb] Adding a new decorator for CMAKE_BUILD_TYPE. (#141159)John Harrison1-0/+3
In lldb-dap, we have existing tests that are known to be unstable when lldb and lldb-dap are built in the Debug configuration. This decorator lets us skip those tests in CI jobs that are to slow with those configurations. This was split out from #140777 to make the patches smaller.
2025-04-01[lldb] add --platform-available-ports option to the dotest.py (#112555)dlav-sc1-0/+1
This patch adds --platform-available-ports option to the dotest.py script to avoid hardcoded gdb ports in lldb testsuite. Currently, this option could be helpful in GdbRemoteTestCases (e.g. TestLldbGdbServer, TestNonStop, TestGdbRemoteThreadsInStopReply, TestGdbRemotePlatformFile, TestGdbRemote_vCont)
2024-10-30Fix a couple of tests that were incorrectly using ↵jimingham1-0/+4
configuration.dwarf_version (#114161) The tests were using the variable directly to get the dwarf version used for the test. That's only the overridden value, and won't be set if we're using the compiler default. I also put a comment by the variable to make sure people don't make the same mistake in the future.
2024-09-25[lldb][test] Use tools from llvm instead of compiler tools (#109961)Vladislav Dzhidzhoev1-0/+3
In #102185, toolchain detection for API tests has been rewritten in Python. Tools paths for tests there are determined from compiler path. Here tools are taken from `--llvm-tools-dir` dotest.py argument, which by default refers to the LLVM build directory, unless they are explicitly redefined in environment variables. It helps to minimize external dependencies and to maximize the reproducibility of the build.
2024-06-03[lldb][test] Add --make argument to dotest.py (#93883)Vladislav Dzhidzhoev1-0/+1
This argument allows to specify the path to make which is used by LLDB API tests to compile test programs. It might come in handy for setting up cross-platform remote runs of API tests on Windows host. It can be used to override the make path of LLDB API tests using `LLDB_TEST_USER_ARGS` argument: ``` cmake ... -DLLDB_TEST_USER_ARGS="...;--make;C:\\Path\\to\\make.exe;..." ... ```
2024-04-29Add a new SBExpressionOptions::SetLanguage() API (NFCI) (#89981)Adrian Prantl1-0/+1
that separates out language and version. To avoid reinventing the wheel and introducing subtle incompatibilities, this API uses the table of languages and versiond defined by the upcoming DWARF 6 standard (https://dwarfstd.org/languages-v6.html). While the DWARF 6 spec is not finialized, the list of languages is broadly considered stable. The primary motivation for this is to allow the Swift language plugin to switch between language dialects between, e.g., Swift 5.9 and 6.0 with out introducing a ton of new language codes. On the main branch this change is considered NFC. Depends on https://github.com/llvm/llvm-project/pull/89980
2024-02-13[lldb][test] Switch LLDB API tests from vendored unittest2 to unittest (#79945)Jordan Rupprecht1-2/+2
This removes the dependency LLDB API tests have on lldb/third_party/Python/module/unittest2, and instead uses the standard one provided by Python. This does not actually remove the vendored dep yet, nor update the docs. I'll do both those once this sticks. Non-trivial changes to call out: - expected failures (i.e. "bugnumber") don't have a reason anymore, so those params were removed - `assertItemsEqual` is now called `assertCountEqual` - When a test is marked xfail, our copy of unittest2 considers failures during teardown to be OK, but modern unittest does not. See TestThreadLocal.py. (Very likely could be a real bug/leak). - Our copy of unittest2 was patched to print all test results, even ones that don't happen, e.g. `(5 passes, 0 failures, 1 errors, 0 skipped, ...)`, but standard unittest prints a terser message that omits test result types that didn't happen, e.g. `OK (skipped=1)`. Our lit integration parses this stderr and needs to be updated w/ that expectation. I tested this w/ `ninja check-lldb-api` on Linux. There's a good chance non-Linux tests have similar quirks, but I'm not able to uncover those.
2023-08-14[lldb] Remove use of __future__ in pythonAlex Langford1-3/+0
These were useful primarily for the Python 2 to 3 transition. Python 2 is no longer supported so these are no longer necessary. Differential Revision: https://reviews.llvm.org/D157759
2023-05-25[NFC][Py Reformat] Reformat python files in lldbJonas Devlieghere1-5/+8
This is an ongoing series of commits that are reformatting our Python code. Reformatting is done with `black` (23.1.0). If you end up having problems merging this commit because you have made changes to a python file, the best way to handle that is to run `git checkout --ours <yourfile>` and then reformat it with black. RFC: https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style Differential revision: https://reviews.llvm.org/D151460
2022-10-26[test] Fix LLDB tests with just-built libcxx when using a target directory.Jordan Rupprecht1-0/+1
In certain configurations, libc++ headers all exist in the same directory, and libc++ binaries exist in the same directory as lldb libs. When `LLVM_ENABLE_PER_TARGET_RUNTIME_DIR` is enabled (*and* the host is not Apple, which is why I assume this wasn't caught by others?), this is not the case: most headers will exist in the usual `include/c++/v1` directory, but `__config_site` exists in `include/$TRIPLE/c++/v1`. Likewise, the libc++.so binary exists in `lib/$TRIPLE/`, not `lib/` (where LLDB libraries reside). This also adds the just-built-libcxx functionality to the lldb-dotest tool. The `LIBCXX_` cmake config is borrowed from `libcxx/CMakeLists.txt`. I could not figure out a way to share the cmake config; ideally we would reuse the same config instead of copy/paste. Reviewed By: JDevlieghere, fdeazeve Differential Revision: https://reviews.llvm.org/D133973
2022-08-22[lldb] Remove prefer-dynamic-value test overrideDave Lee1-2/+1
Remove the test override of `target.prefer-dynamic-value`. Previously, the lldb default was `no-dynamic-values`. In rG9aa7e8e9ffbe (in 2015), the default was changed to `no-run-target`, but at that time the tests were changed to be run with `no-dynamic-value`. I don't know the reasons for not changing the tests, perhaps to avoid determining which tests to change, and what about them to change. Because `no-run-target` is the lldb default, I think it makes sense to make it the test default too. It puts the test config closer to what's used in practice. This change removes the `target.prefer-dynamic-value` override, and for those tests that failed, they have been updated to explicitly use `no-dynamic-values`. Future changes could update these tests to use dynamic values too, or they can be left as is to exercise non-dynamic typing. Differential Revision: https://reviews.llvm.org/D132382
2022-08-19[lldb] Support specifying a custom libcxx for the API testsJonas Devlieghere1-2/+2
This patch combines D129166 (to always pick the just-built libc++) and D132257 (to allow customizing the libc++ for testing). The common goal is to avoid picking up an unexpected libc++ for testing. Differential revision: https://reviews.llvm.org/D132263
2022-07-11[lldb] Use the just-built libc++ for testing the LLDB data formattersJonas Devlieghere1-0/+3
Make sure we use the libc++ from the build dir. Currently, by passing -stdlib=libc++, we might pick up the system libc++. This change ensures that if LLVM_LIBS_DIR is set, we try to use the libc++ from there. Differential revision: https://reviews.llvm.org/D129166
2021-09-30[lldb] Remove support for replaying the test suite from a reproducerJonas Devlieghere1-19/+0
This patch removes the infrastructure to replay the test suite from a reproducer, as well as the modifications made to the individual tests.
2021-07-20Remove the DarwinLog functionality from debguserverJason Molenda1-1/+1
Remove the DarwinLog and qStructuredDataPlugins support from debugserver. The DarwinLog plugin was never debugged fully and made reliable, and the underlying private APIs it uses have migrated since 2016 so none of them exist any longer. Differential Revision: https://reviews.llvm.org/D106324 rdar://75073283
2020-12-04[lldb] Remove LLDB session dir and just store test traces in the respective ↵Raphael Isemann1-14/+0
test build directory Test runs log some of their output to files inside the LLDB session dir. This session dir is shared between all tests, so all the tests have to make sure they choose a unique file name inside that directory. We currently choose by default `<test-class-name>-<test-method-name>` as the log file name. However, that means that if not every test class in the test suite has a unique class name, then we end up with a race condition as two tests will try to write to the same log file. I already tried in D83767 changing the format to use the test file basename instead (which we already require to be unique for some other functionality), but it seems the code for getting the basename didn't work on Windows. This patch instead just changes that dotest stores the log files in the build directory for the current test. We know that directory is unique for this test, so no need to generate some unique file name now. Also removes all the environment vars and parameters related to the now unused session dir. The new log paths now look like this for a failure in 'TestCppOperators`: ``` ./lldb-test-build.noindex/lang/cpp/operators/TestCppOperators.test_dwarf/Failure.log ./lldb-test-build.noindex/lang/cpp/operators/TestCppOperators.test_dsym/Failure.log ./lldb-test-build.noindex/lang/cpp/operators/TestCppOperators.test_gmodules/Failure.log ``` Reviewed By: labath Differential Revision: https://reviews.llvm.org/D92498
2020-08-21[lldb] Remove --rerun-all-issues as its functionality no longer existsJonas Devlieghere1-3/+0
The logic behind --rerun-all-issues was removed when we switched to LIT as the test driver. This patch just removes the dotest option and corresponding entry in configuration.py.
2020-08-07[lldb] Store the Apple SDK in dotest's configuration.Jonas Devlieghere1-0/+3
This patch stores the --apple-sdk argument in the dotest configuration. When it's set, use it instead of the triple to determine the current platform. Differential revision: https://reviews.llvm.org/D85537
2020-08-05[lldb/test] Replace LLDB_TEST_SRC env variable with configurationJordan Rupprecht1-0/+4
Reviewed By: JDevlieghere Differential Revision: https://reviews.llvm.org/D85322
2020-07-13[lldb/dotest] Remove the "xunit" result formatterPavel Labath1-4/+0
Summary: My understanding is that this was added to make dotest interact well with the GreenDragon bots, back when dotest was the main test driver. Now that everything goes through lit (which has its own xunit formatter), it seems largely irrelevant. There are more cleanups that can be done after removing this be done here, but this should be enough to test the waters. Reviewers: JDevlieghere Subscribers: lldb-commits Tags: #lldb Differential Revision: https://reviews.llvm.org/D83545
2020-07-10[lldb/Test] Fix missing yaml2obj in Xcode standalone build.Jonas Devlieghere1-0/+10
Rather than trying to find the yaml2obj from dotest we should pass it in like we do for dsymutil and FileCheck.
2020-06-02[lldb/Test] Don't use the env to pass around configuration variables (NFC)Jonas Devlieghere1-2/+4
Don't use the environment to pass values to the builder. Use the configuration instead.
2020-05-14[lldb/Test] Modify more tests for API replayJonas Devlieghere1-1/+10
Skip tests or parts thereof that aren't expected to work when run from a reproducer. Also improve the doc comments in configuration.py to prevent mistakes in the future.
2020-05-08[lldb/Test] Update TestProcessList.py for reproducer replayJonas Devlieghere1-0/+6
Because LLDB isn't the one spawning the subprocess, the PID is different during replay. Exclude it form the substring check during replay. Depends on D79646 to pass with reproducer replay.
2020-04-15Another attempt of D77452 - da0e91fee614e8686f48db28e6507c8abe061fc2Walter Erquinigo1-0/+3
[intel-pt] Improve the way the test determines whether to run - Now I'm creating a default value for the new test parameter - I fixed a small mistake in the skipping logic of the test ... I forgot to clear the cmake cache when testing my diff
2020-04-15Revert "[intel-pt] Improve the way the test determines whether to run"Walter Erquinigo1-3/+0
This reverts commit da0e91fee614e8686f48db28e6507c8abe061fc2. There's a failure in http://lab.llvm.org:8011/builders/lldb-x86_64-debian/builds/8584 caused by a missing python object.
2020-04-15[intel-pt] Improve the way the test determines whether to runWalter Erquinigo1-0/+3
Summary: @labath raised a concern on the way I was skipping this test. I think that was fair and I found a better way. Now I'm skipping if the CMAKE flag LLDB_BUILD_INTEL_PT is false. I added an enabled_plugins entry in the dotest configuration, which gets set by lit or lldb-dotest. The only available plugin right now is 'intel-pt', but I imagine it will be useful in the future for other kinds of plugins that get determined at configuration time. I didn't want to add a new argument option --enable-intel-pt or something or the sort, as it wouldn't be useful for other cases. Reviewers: labath, clayborg Subscribers: lldb-commits, labath Tags: #lldb Differential Revision: https://reviews.llvm.org/D77452
2020-04-14[lldb/Reproducers] Capture reproducers from the API test suite.Jonas Devlieghere1-1/+4
Make it possible to capture reproducers from the API test suite. Given the symmetry between capture and replay, this patch also adds the necessary code for replay. For now this is a NO-OP until the corresponding reproducer instrumentation changes land. For more info please refer to the RFC on lldb-dev: http://lists.llvm.org/pipermail/lldb-dev/2020-April/016100.html Differential revision: https://reviews.llvm.org/D77588
2020-02-14[lldb/dotest] Remove the "exclusive test subdir" conceptPavel Labath1-36/+0
Summary: This was added in 2018 (r339929), when we were still using the hand-rolled test runner. It does not seem to be relevant anymore. In fact as far as I can tell, it's a big no-op now as the exclusive_test_subdir variable is never set. Reviewers: vsk, JDevlieghere Subscribers: lldb-commits Tags: #lldb Differential Revision: https://reviews.llvm.org/D74551
2020-02-13[lldb] Fix lldb-dotestPavel Labath1-1/+1
to account for the new location of test files from 99451b445.
2020-02-05[lldb] [test] Pass LLVM_LIBS_DIR from CMake for linking liblldbMichał Górny1-0/+4
Pass the correct library directory from CMake to dotest.py when linking liblldb, instead of trying to reconstruct the path from executable path. This fixes link failures on platforms having non-null LLVM_LIBDIR_SUFFIX. Differential Revision: https://reviews.llvm.org/D73767
2020-01-14dotest.py: Add option to pass extra lldb settings to dotestAdrian Prantl1-0/+4
The primary motivation for this is to add another dimension to the Swift LLDB test matrix, but this seems generally useful. Differential Revision: https://reviews.llvm.org/D72662
2020-01-10[lldb][tests] Make it possible to expect failure for a whole categoryTatyana Krasnukha1-0/+2
There already are decorators and "--excluded" option to mark test-cases/files as expected to fail. However, when a new test file is added and it which relates to a feature that a target doesn't support, this requires either adding decorators to that file or modifying the file provided as "--excluded" option value. The purpose of this patch is to avoid any modifications in such cases. E.g. if a target doesn't support "watchpoints" and passes "--xfail-category watchpoint" to dotest, a testing job will not fail after a new watchpoint-related test file is added. Differential Revision: https://reviews.llvm.org/D71906
2020-01-10[lldb][tests][NFC] Unify variable naming conventionTatyana Krasnukha1-8/+8
2019-10-10[test] Use a different module cache for Shell and API tests.Jonas Devlieghere1-1/+3
Before the test reorganization, everything was part of a single test suite with a single module cache. Now that things are properly separated this is no longer the case. Only the shell tests inherited the logic to properly configure and wipe the module caches. This patch adds that logic back for the API tests. While doing so, I noticed that we were configuring a Clang module cache in CMake, but weren't actually using it from dotest.py. I included a fix for that in this patch as well. Differential revision: https://reviews.llvm.org/D68755 llvm-svn: 374386
2019-08-29[test] Fix various module cache bugs and inconsistenciesJonas Devlieghere1-0/+3
Currently, lit tests don't set neither the module cache for building inferiors nor the module cache used by lldb when running tests. Furthermore, we have several places where we rely on the path to the module cache being always the same, rather than passing the correct value around. This makes it hard to specify a different module cache path when debugging a a test. This patch reworks how we determine and pass around the module cache paths and fixes the omission on the lit side. It also adds a sanity check to the lit and dotest suites. Differential revision: https://reviews.llvm.org/D66966 llvm-svn: 370394
2019-08-28[dotest] Remove --rerun-max-file-thresholdJonas Devlieghere1-1/+0
This variable corresponding to this argument is set but never read. llvm-svn: 370264
2019-08-28[dotest] Remove -q (quiet) flag.Jonas Devlieghere1-4/+0
This patch removes the -q (quiet) flag and changing the default behavior. Currently the flag serves two purposes that are somewhat contradictory, as illustrated by the difference between the argument name (quiet) and the configuration flag (parsable). On the one hand it reduces output, but on the other hand it prints more output, like the result of individual tests. My proposal is to guard the extra output behind the verbose flag and always print the individual test results. Differential revision: https://reviews.llvm.org/D66837 llvm-svn: 370226
2019-08-27[dotest] Remove results portJonas Devlieghere1-1/+0
The results port was used by dosep.py to deal with test results coming form different processes. With dosep.py gone, I don't think we need this any longer. Differential revision: https://reviews.llvm.org/D66811 llvm-svn: 370090
2019-08-27[dotest] Remove long running test "decorator" and re-enable tests.Jonas Devlieghere1-3/+0
Today I discovered the skipLongRunningTest decorator and to my surprise all the tests were passing without the decorator. They don't seem to be that expensive either, they take a few seconds but we have tests that take much longer than that. As such I propose to remove the decorator and enable them by default. Differential revision: https://reviews.llvm.org/D66774 llvm-svn: 369995
2019-08-19[dotest] Add --dwarf-version to override the tested DWARF version.Jonas Devlieghere1-0/+3
On the matrix bot on GreenDragon [1] we want to run the test suite against different DWARF versions. The idea here is not to replace targeted tests for certain DWARF features, but rather to provide an easy way to support this configuration. [1] http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake-matrix/ Differential revision: https://reviews.llvm.org/D66370 llvm-svn: 369272
2019-08-02[lldb][NFC] Remove unused imports in python testsRaphael Isemann1-2/+0
llvm-svn: 367663
2019-07-30[dotest] Remove multiprocessingJonas Devlieghere1-6/+0
Now that the Xcode project is removed, I want to focus on dotest as a test framework, and remove its driver capabilities for which we already rely on llvm's lit. Removing multiprocessing is the first step in that direction. Differential revision: https://reviews.llvm.org/D65311 llvm-svn: 367331
2019-01-19Update the file headers across all of the LLVM projects in the monorepoChandler Carruth1-4/+3
to reflect the new license. We understand that people may be surprised that we're moving the header entirely to discuss the new license. We checked this carefully with the Foundation's lawyer and we believe this is the correct approach. Essentially, all code in the project is now made available by the LLVM project under our new license, so you will see that the license headers include that license only. Some of our contributors have contributed code under our old license, and accordingly, we have retained a copy of our old license notice in the top-level files in each project and repository. llvm-svn: 351636
2018-10-12Fix failure in get_filecheck_path when filecheck is NoneStella Stamenova1-1/+1
If the path was not specified (and it's None), lexists throws an exception rather than returning False. get_filecheck_path now checks whether filecheck is set before calling lexists llvm-svn: 344410
2018-10-12[dotest] Make a missing FileCheck binary a warning, not an errorVedant Kumar1-2/+2
This allows bots which haven't updated to pass in --filecheck to dotest.py to run more tests. FileCheck-dependent tests will continue to fail. Differential Revision: https://reviews.llvm.org/D53175 llvm-svn: 344401
2018-09-18Allow use of self.filecheck in LLDB tests (c.f self.expect)Vedant Kumar1-0/+11
Add a "filecheck" method to the LLDB test base. This allows test authors to pattern match command output using FileCheck, making it possible to write stricter tests than what `self.expect` allows. For context (motivation, examples of stricter checking, etc), see the lldb-dev thread: "Using FileCheck in lldb inline tests". Differential Revision: https://reviews.llvm.org/D50751 llvm-svn: 342508
2018-08-16[dotest] Make --test-subdir work with --no-multiprocessVedant Kumar1-1/+36
The single-process test runner is invoked in a number of different scenarios, including when multiple test dirs are specified or (afaict) when lit is used to drive the test suite. Unfortunately the --test-subdir option did not work with the single process test runner, breaking an important use case (using lit to run swift-lldb Linux tests): Failure URL: https://ci.swift.org/job/swift-PR-Linux/6841 We won't be able to run lldb tests within swift PR testing without filtering down the set of tests. This change makes --test-subdir work with the single-process runner. llvm-svn: 339929