Age | Commit message (Collapse) | Author | Files | Lines |
|
I've wanted a utility to create a corefile for test purposes given a bit
of memory and regsters, for a while. I've written a few API tests over
the years that needed exactly this capability -- we have several one-off
Mach-O corefile creator utility in the API testsuite to do this. But
it's a lot of boilerplate when you only want to specify some register
contents and memory contents, to create an API test.
This adds yaml2mach-core, a tool that should build on any system, takes
a yaml description of register values for one or more threads,
optionally memory values for one or more memory regions, and can take a
list of UUIDs that will be added as LC_NOTE "load binary" metadata to
the corefile so binaries can be loaded into virtual address space in a
test scenario.
The format of the yaml file looks like
```
cpu: armv7m
# optionally specify the number of bits used for addressing
# (this line is from a different, 64-bit, yaml file)
addressable-bits:
num-bits: 39
# optionally specify one or more binary UUID and slide/virtual address to be added as an LC_NOTE
# (this line is from a different, 64-bit, yaml file)
binaries:
- name: debug-binary.development
uuid: 67942352-5857-3D3D-90CB-A3F80BA67B04
virtual-address: 0xfffffff01840c000
threads:
- regsets:
- flavor: gpr
registers: [{name: sp, value: 0x2000fe70}, {name: r7, value: 0x2000fe80},
{name: pc, value: 0x0020392c}, {name: lr, value: 0x0020392d}]
memory-regions:
# stack memory
- addr: 0x2000fe70
UInt32: [ 0x0000002a, 0x20010e58, 0x00203923,
0x00000001, 0x2000fe88, 0x00203911,
0x2000ffdc, 0xfffffff9 ]
# instructions of a function
- addr: 0x203910
UInt8: [ 0xf8, 0xb5, 0x04, 0xaf, 0x06, 0x4c, 0x07, 0x49,
0x74, 0xf0, 0x2e, 0xf8, 0x01, 0xac, 0x74, 0xf0 ]
```
and that's all that is needed to specify a corefile where four register
values are specified (the others will be set to 0), and two memory
regions will be emitted.
The memory can be specified as an array of UInt8, UInt32, or UInt64, I
anticipate that some of these corefiles may have stack values
constructed manually and it may be simpler for a human to write them in
a particular grouping of values.
I needed this utility for an upcoming patch for ARM Cortex-M processors,
to create a test for the change. I took the opportunity to remove two of
the "trivial mach-o corefile" creator utilities I've written in the
past, which also restricted the tests to only run on Darwin systems
because I was using the system headers for Mach-O constant values.
rdar://110663219
|
|
(#151403)
Fix cross-compilation of test inferiors on Darwin, targeting remote
Linux. This requires specifying the target triple and using LLD for
linking.
Fixes #150806
|
|
In lldb-dap, we have existing tests that are known to be unstable when
lldb and lldb-dap are built in the Debug configuration.
This decorator lets us skip those tests in CI jobs that are to slow with
those configurations.
This was split out from #140777 to make the patches smaller.
|
|
This patch adds --platform-available-ports option to the dotest.py
script to avoid hardcoded gdb ports in lldb testsuite.
Currently, this option could be helpful in GdbRemoteTestCases (e.g.
TestLldbGdbServer, TestNonStop, TestGdbRemoteThreadsInStopReply,
TestGdbRemotePlatformFile, TestGdbRemote_vCont)
|
|
configuration.dwarf_version (#114161)
The tests were using the variable directly to get the dwarf version used
for the test. That's only the overridden value, and won't be set if
we're using the compiler default. I also put a comment by the variable
to make sure people don't make the same mistake in the future.
|
|
In #102185, toolchain detection for API tests has been rewritten in
Python. Tools paths for tests there are determined from compiler path.
Here tools are taken from `--llvm-tools-dir` dotest.py argument, which
by default refers to the LLVM build directory, unless they are
explicitly redefined in environment variables. It helps to minimize
external dependencies and to maximize the reproducibility of the build.
|
|
This argument allows to specify the path to make which is used by
LLDB API tests to compile test programs.
It might come in handy for setting up cross-platform remote runs of API tests on Windows host.
It can be used to override the make path of LLDB API tests using `LLDB_TEST_USER_ARGS` argument:
```
cmake ...
-DLLDB_TEST_USER_ARGS="...;--make;C:\\Path\\to\\make.exe;..."
...
```
|
|
that separates out language and version. To avoid reinventing the wheel
and introducing subtle incompatibilities, this API uses the table of
languages and versiond defined by the upcoming DWARF 6 standard
(https://dwarfstd.org/languages-v6.html). While the DWARF 6 spec is not
finialized, the list of languages is broadly considered stable.
The primary motivation for this is to allow the Swift language plugin to
switch between language dialects between, e.g., Swift 5.9 and 6.0 with
out introducing a ton of new language codes. On the main branch this
change is considered NFC.
Depends on https://github.com/llvm/llvm-project/pull/89980
|
|
This removes the dependency LLDB API tests have on
lldb/third_party/Python/module/unittest2, and instead uses the standard
one provided by Python.
This does not actually remove the vendored dep yet, nor update the docs.
I'll do both those once this sticks.
Non-trivial changes to call out:
- expected failures (i.e. "bugnumber") don't have a reason anymore, so
those params were removed
- `assertItemsEqual` is now called `assertCountEqual`
- When a test is marked xfail, our copy of unittest2 considers failures
during teardown to be OK, but modern unittest does not. See
TestThreadLocal.py. (Very likely could be a real bug/leak).
- Our copy of unittest2 was patched to print all test results, even ones
that don't happen, e.g. `(5 passes, 0 failures, 1 errors, 0 skipped,
...)`, but standard unittest prints a terser message that omits test
result types that didn't happen, e.g. `OK (skipped=1)`. Our lit
integration parses this stderr and needs to be updated w/ that
expectation.
I tested this w/ `ninja check-lldb-api` on Linux. There's a good chance
non-Linux tests have similar quirks, but I'm not able to uncover those.
|
|
These were useful primarily for the Python 2 to 3 transition. Python 2
is no longer supported so these are no longer necessary.
Differential Revision: https://reviews.llvm.org/D157759
|
|
This is an ongoing series of commits that are reformatting our Python
code. Reformatting is done with `black` (23.1.0).
If you end up having problems merging this commit because you have made
changes to a python file, the best way to handle that is to run `git
checkout --ours <yourfile>` and then reformat it with black.
RFC: https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style
Differential revision: https://reviews.llvm.org/D151460
|
|
In certain configurations, libc++ headers all exist in the same directory, and libc++ binaries exist in the same directory as lldb libs. When `LLVM_ENABLE_PER_TARGET_RUNTIME_DIR` is enabled (*and* the host is not Apple, which is why I assume this wasn't caught by others?), this is not the case: most headers will exist in the usual `include/c++/v1` directory, but `__config_site` exists in `include/$TRIPLE/c++/v1`. Likewise, the libc++.so binary exists in `lib/$TRIPLE/`, not `lib/` (where LLDB libraries reside).
This also adds the just-built-libcxx functionality to the lldb-dotest tool.
The `LIBCXX_` cmake config is borrowed from `libcxx/CMakeLists.txt`. I could not figure out a way to share the cmake config; ideally we would reuse the same config instead of copy/paste.
Reviewed By: JDevlieghere, fdeazeve
Differential Revision: https://reviews.llvm.org/D133973
|
|
Remove the test override of `target.prefer-dynamic-value`.
Previously, the lldb default was `no-dynamic-values`. In rG9aa7e8e9ffbe (in
2015), the default was changed to `no-run-target`, but at that time the tests
were changed to be run with `no-dynamic-value`. I don't know the reasons for
not changing the tests, perhaps to avoid determining which tests to change, and
what about them to change.
Because `no-run-target` is the lldb default, I think it makes sense to make it
the test default too. It puts the test config closer to what's used in
practice.
This change removes the `target.prefer-dynamic-value` override, and for those
tests that failed, they have been updated to explicitly use
`no-dynamic-values`. Future changes could update these tests to use dynamic
values too, or they can be left as is to exercise non-dynamic typing.
Differential Revision: https://reviews.llvm.org/D132382
|
|
This patch combines D129166 (to always pick the just-built libc++) and
D132257 (to allow customizing the libc++ for testing). The common goal
is to avoid picking up an unexpected libc++ for testing.
Differential revision: https://reviews.llvm.org/D132263
|
|
Make sure we use the libc++ from the build dir. Currently, by passing
-stdlib=libc++, we might pick up the system libc++. This change ensures
that if LLVM_LIBS_DIR is set, we try to use the libc++ from there.
Differential revision: https://reviews.llvm.org/D129166
|
|
This patch removes the infrastructure to replay the test suite from a
reproducer, as well as the modifications made to the individual tests.
|
|
Remove the DarwinLog and qStructuredDataPlugins support
from debugserver. The DarwinLog plugin was never debugged
fully and made reliable, and the underlying private APIs
it uses have migrated since 2016 so none of them exist
any longer.
Differential Revision: https://reviews.llvm.org/D106324
rdar://75073283
|
|
test build directory
Test runs log some of their output to files inside the LLDB session dir. This
session dir is shared between all tests, so all the tests have to make sure they
choose a unique file name inside that directory. We currently choose by default
`<test-class-name>-<test-method-name>` as the log file name. However, that means
that if not every test class in the test suite has a unique class name, then we
end up with a race condition as two tests will try to write to the same log
file.
I already tried in D83767 changing the format to use the test file basename
instead (which we already require to be unique for some other functionality),
but it seems the code for getting the basename didn't work on Windows.
This patch instead just changes that dotest stores the log files in the build
directory for the current test. We know that directory is unique for this test,
so no need to generate some unique file name now. Also removes all the
environment vars and parameters related to the now unused session dir.
The new log paths now look like this for a failure in 'TestCppOperators`:
```
./lldb-test-build.noindex/lang/cpp/operators/TestCppOperators.test_dwarf/Failure.log
./lldb-test-build.noindex/lang/cpp/operators/TestCppOperators.test_dsym/Failure.log
./lldb-test-build.noindex/lang/cpp/operators/TestCppOperators.test_gmodules/Failure.log
```
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D92498
|
|
The logic behind --rerun-all-issues was removed when we switched to LIT
as the test driver. This patch just removes the dotest option and
corresponding entry in configuration.py.
|
|
This patch stores the --apple-sdk argument in the dotest configuration.
When it's set, use it instead of the triple to determine the current
platform.
Differential revision: https://reviews.llvm.org/D85537
|
|
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D85322
|
|
Summary:
My understanding is that this was added to make dotest interact well
with the GreenDragon bots, back when dotest was the main test driver.
Now that everything goes through lit (which has its own xunit
formatter), it seems largely irrelevant.
There are more cleanups that can be done after removing this be done
here, but this should be enough to test the waters.
Reviewers: JDevlieghere
Subscribers: lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D83545
|
|
Rather than trying to find the yaml2obj from dotest we should pass it in
like we do for dsymutil and FileCheck.
|
|
Don't use the environment to pass values to the builder. Use the
configuration instead.
|
|
Skip tests or parts thereof that aren't expected to work when run from a
reproducer. Also improve the doc comments in configuration.py to prevent
mistakes in the future.
|
|
Because LLDB isn't the one spawning the subprocess, the PID is different
during replay. Exclude it form the substring check during replay.
Depends on D79646 to pass with reproducer replay.
|
|
[intel-pt] Improve the way the test determines whether to run
- Now I'm creating a default value for the new test parameter
- I fixed a small mistake in the skipping logic of the test
... I forgot to clear the cmake cache when testing my diff
|
|
This reverts commit da0e91fee614e8686f48db28e6507c8abe061fc2.
There's a failure in
http://lab.llvm.org:8011/builders/lldb-x86_64-debian/builds/8584
caused by a missing python object.
|
|
Summary:
@labath raised a concern on the way I was skipping this test. I think that was
fair and I found a better way.
Now I'm skipping if the CMAKE flag LLDB_BUILD_INTEL_PT is false.
I added an enabled_plugins entry in the dotest configuration, which gets
set by lit or lldb-dotest. The only available plugin right now is
'intel-pt', but I imagine it will be useful in the future for other
kinds of plugins that get determined at configuration time. I didn't
want to add a new argument option --enable-intel-pt or something or the
sort, as it wouldn't be useful for other cases.
Reviewers: labath, clayborg
Subscribers: lldb-commits, labath
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D77452
|
|
Make it possible to capture reproducers from the API test suite. Given
the symmetry between capture and replay, this patch also adds the
necessary code for replay. For now this is a NO-OP until the
corresponding reproducer instrumentation changes land.
For more info please refer to the RFC on lldb-dev:
http://lists.llvm.org/pipermail/lldb-dev/2020-April/016100.html
Differential revision: https://reviews.llvm.org/D77588
|
|
Summary:
This was added in 2018 (r339929), when we were still using the
hand-rolled test runner.
It does not seem to be relevant anymore. In fact as far as I can tell,
it's a big no-op now as the exclusive_test_subdir variable is never set.
Reviewers: vsk, JDevlieghere
Subscribers: lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D74551
|
|
to account for the new location of test files from 99451b445.
|
|
Pass the correct library directory from CMake to dotest.py when linking
liblldb, instead of trying to reconstruct the path from executable path.
This fixes link failures on platforms having non-null
LLVM_LIBDIR_SUFFIX.
Differential Revision: https://reviews.llvm.org/D73767
|
|
The primary motivation for this is to add another dimension to the
Swift LLDB test matrix, but this seems generally useful.
Differential Revision: https://reviews.llvm.org/D72662
|
|
There already are decorators and "--excluded" option to mark test-cases/files
as expected to fail. However, when a new test file is added and it which relates
to a feature that a target doesn't support, this requires either adding decorators
to that file or modifying the file provided as "--excluded" option value.
The purpose of this patch is to avoid any modifications in such cases.
E.g. if a target doesn't support "watchpoints" and passes "--xfail-category watchpoint"
to dotest, a testing job will not fail after a new watchpoint-related test file is added.
Differential Revision: https://reviews.llvm.org/D71906
|
|
|
|
Before the test reorganization, everything was part of a single test
suite with a single module cache. Now that things are properly separated
this is no longer the case. Only the shell tests inherited the logic to
properly configure and wipe the module caches. This patch adds that
logic back for the API tests. While doing so, I noticed that we were
configuring a Clang module cache in CMake, but weren't actually using it
from dotest.py. I included a fix for that in this patch as well.
Differential revision: https://reviews.llvm.org/D68755
llvm-svn: 374386
|
|
Currently, lit tests don't set neither the module cache for building
inferiors nor the module cache used by lldb when running tests.
Furthermore, we have several places where we rely on the path to the
module cache being always the same, rather than passing the correct
value around. This makes it hard to specify a different module cache
path when debugging a a test.
This patch reworks how we determine and pass around the module cache
paths and fixes the omission on the lit side. It also adds a sanity
check to the lit and dotest suites.
Differential revision: https://reviews.llvm.org/D66966
llvm-svn: 370394
|
|
This variable corresponding to this argument is set but never read.
llvm-svn: 370264
|
|
This patch removes the -q (quiet) flag and changing the default
behavior. Currently the flag serves two purposes that are somewhat
contradictory, as illustrated by the difference between the argument
name (quiet) and the configuration flag (parsable). On the one hand it
reduces output, but on the other hand it prints more output, like the
result of individual tests. My proposal is to guard the extra output
behind the verbose flag and always print the individual test results.
Differential revision: https://reviews.llvm.org/D66837
llvm-svn: 370226
|
|
The results port was used by dosep.py to deal with test results coming
form different processes. With dosep.py gone, I don't think we need this
any longer.
Differential revision: https://reviews.llvm.org/D66811
llvm-svn: 370090
|
|
Today I discovered the skipLongRunningTest decorator and to my surprise
all the tests were passing without the decorator. They don't seem to be
that expensive either, they take a few seconds but we have tests that
take much longer than that. As such I propose to remove the decorator
and enable them by default.
Differential revision: https://reviews.llvm.org/D66774
llvm-svn: 369995
|
|
On the matrix bot on GreenDragon [1] we want to run the test suite
against different DWARF versions. The idea here is not to replace
targeted tests for certain DWARF features, but rather to provide an easy
way to support this configuration.
[1] http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake-matrix/
Differential revision: https://reviews.llvm.org/D66370
llvm-svn: 369272
|
|
llvm-svn: 367663
|
|
Now that the Xcode project is removed, I want to focus on dotest as a
test framework, and remove its driver capabilities for which we already
rely on llvm's lit. Removing multiprocessing is the first step in that
direction.
Differential revision: https://reviews.llvm.org/D65311
llvm-svn: 367331
|
|
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
|
|
If the path was not specified (and it's None), lexists throws an exception rather than returning False. get_filecheck_path now checks whether filecheck is set before calling lexists
llvm-svn: 344410
|
|
This allows bots which haven't updated to pass in --filecheck to
dotest.py to run more tests. FileCheck-dependent tests will continue to
fail.
Differential Revision: https://reviews.llvm.org/D53175
llvm-svn: 344401
|
|
Add a "filecheck" method to the LLDB test base. This allows test authors
to pattern match command output using FileCheck, making it possible to
write stricter tests than what `self.expect` allows.
For context (motivation, examples of stricter checking, etc), see the
lldb-dev thread: "Using FileCheck in lldb inline tests".
Differential Revision: https://reviews.llvm.org/D50751
llvm-svn: 342508
|
|
The single-process test runner is invoked in a number of different
scenarios, including when multiple test dirs are specified or (afaict)
when lit is used to drive the test suite.
Unfortunately the --test-subdir option did not work with the single
process test runner, breaking an important use case (using lit to run
swift-lldb Linux tests):
Failure URL: https://ci.swift.org/job/swift-PR-Linux/6841
We won't be able to run lldb tests within swift PR testing without
filtering down the set of tests.
This change makes --test-subdir work with the single-process runner.
llvm-svn: 339929
|