Age | Commit message (Collapse) | Author | Files | Lines |
|
This patch reverts 44e791c6ff1a982de9651aad7d1c83d1ad96da8a,
3cc1031a827d319c6cb48df1c3aafc9ba7e96d72 and
adbd43250ade1d5357542d8bd7c3dfed212ddec0. Which breaks debuginfod build
and tests when httplib is used.
|
|
This patch fixes:
llvm/lib/Debuginfod/Debuginfod.cpp:592:12: error: moving a local
object in a return statement prevents copy elision
[-Werror,-Wpessimizing-move]
llvm/lib/Debuginfod/Debuginfod.cpp:613:12: error: moving a local
object in a return statement prevents copy elision
[-Werror,-Wpessimizing-move]
|
|
Fix #63873
|
|
|
|
(#136121)
…Stream.
CachedFileStream has previously performed the commit step in its
destructor, but this means its only recourse for error handling is
report_fatal_error. Modify this to add an explicit commit() method, and
call this in the appropriate places with appropriate error handling for
the location.
Currently the destructor of CacheStream gives an assert failure in Debug
builds if commit() was not called. This will help track down any
remaining uses of the API that assume the old destructior behaviour. In
Release builds we fall back to the previous behaviour and call
report_fatal_error if the commit fails.
This is version 2 of this PR, superseding reverted PR
https://github.com/llvm/llvm-project/pull/115331 . I have incorporated a
change to the testcase to make it more reliable on Windows, as well as
two follow-up changes
(https://github.com/llvm/llvm-project/commit/df79000896101acc9b8d7435e59f767b36c00ac8
and
https://github.com/llvm/llvm-project/commit/b0baa1d8bd68a2ce2f7c5f2b62333e410e9122a1)
that were also reverted when 115331 was reverted.
---------
Co-authored-by: Augie Fackler <augie@google.com>
Co-authored-by: Vitaly Buka <vitalybuka@google.com>
|
|
CachedFile… (#115331)"
This reverts commit ce9e1d3c15ed6290f1cb07b482939976fa8115cd.
The unittest added in this commit seems to be flaky causing random failure on buildbots:
- https://lab.llvm.org/buildbot/#/builders/46/builds/13235
- https://lab.llvm.org/buildbot/#/builders/46/builds/13232
- https://lab.llvm.org/buildbot/#/builders/46/builds/13228
- https://lab.llvm.org/buildbot/#/builders/46/builds/13224
- https://lab.llvm.org/buildbot/#/builders/46/builds/13220
- https://lab.llvm.org/buildbot/#/builders/46/builds/13210
- https://lab.llvm.org/buildbot/#/builders/46/builds/13208
- https://lab.llvm.org/buildbot/#/builders/46/builds/13207
- https://lab.llvm.org/buildbot/#/builders/46/builds/13202
- https://lab.llvm.org/buildbot/#/builders/46/builds/13196
and
- https://lab.llvm.org/buildbot/#/builders/180/builds/14266
- https://lab.llvm.org/buildbot/#/builders/180/builds/14254
- https://lab.llvm.org/buildbot/#/builders/180/builds/14250
- https://lab.llvm.org/buildbot/#/builders/180/builds/14245
- https://lab.llvm.org/buildbot/#/builders/180/builds/14244
- https://lab.llvm.org/buildbot/#/builders/180/builds/14226
|
|
This reverts commit df79000896101acc9b8d7435e59f767b36c00ac8.
Reverting follow-up commit to ce9e1d3c15ed6290f1cb07b482939976fa8115cd since the original commit test is flaky.
|
|
|
|
(#115331)
…Stream.
CachedFileStream has previously performed the commit step in its
destructor, but this means its only recourse for error handling is
report_fatal_error. Modify this to add an explicit commit() method, and
call this in the appropriate places with appropriate error handling for
the location.
Currently the destructor of CacheStream gives an assert failure in Debug
builds if commit() was not called. This will help track down any
remaining uses of the API that assume the old destructior behaviour. In
Release builds we fall back to the previous behaviour and call
report_fatal_error if the commit fails.
|
|
(#122816)
This reverts commit 7b808e73aa0193c8a42eae8f2420a803f424bee1.
Previous commit which change default debuginfod cache path and pruning
policy settings is problematic. It broke multiple tests across lldb and
llvm. Reverting for now.
Co-authored-by: George Hu <georgehuyubo@gmail.com>
|
|
This PR include two changes:
1. Change debuginfod cache file name to include origin file name, the
new file name would be something like:
llvmcache-13267c5f5d2e3df472c133c8efa45fb3331ef1ea-liblzma.so.5.2.2.debuginfo.dwp
So it will provide more information in image list instead of a plain
llvmcache-123
2. Switch debuginfod cache to use lldb index cache settings. Currently
we don't have proper settings for setting the cache path or the cache
expiration time for debuginfod cache. We want to use the lldb index
cache settings, as they make sense to be in the same place and have the
same TTL.
---------
Co-authored-by: George Hu <georgehuyubo@gmail.com>
|
|
(#82094)
This decouples the public API used to enqueue tasks and wait for
completion from the actual implementation, and opens up the possibility
for clients to set their own thread pool implementation for the pool.
https://discourse.llvm.org/t/construct-threadpool-from-vector-of-existing-threads/76883
|
|
This is addressing a long-time TODO to rename this misleading API. The
old one is preserved for now but marked deprecated.
|
|
I've been working on more/better configuration for improving DEBUGINFOD
support. This is the first (and easiest) slice of the work.
I've added `timeout` and `cache-path` settings that can override the
DEBUGINFOD library defaults (and environment variables.) I also renamed
the `plugin.symbol-locator.debuginfod.server_urls` setting to
`server-urls` to be more consistent with the rest of LLDB's settings
(the underscore switch is switched to a hyphen)
I've got a few tests that validate the cache-path setting (as a
side-effect), but they've exposed a few bugs that I'll be putting up a
separate PR for (which will include the tests).
---------
Co-authored-by: Kevin Frei <freik@meta.com>
|
|
@nico pointed out that my usage of `std::shared_mutex` broke builds on
older macOS devices. Switching to `llvm::sys::RWMutex` is the solution
that they provided.
Tracked in issue #74382
Co-authored-by: Kevin Frei <freik@meta.com>
|
|
I've plumbed the LLVM DebugInfoD client into LLDB, and added automatic
downloading of DWP files to the SymbolFileDWARF.cpp plugin. If you have
DEBUGINFOD_URLS set to a space delimited set of web servers, LLDB will
try to use them as a last resort when searching for DWP files. If you do
*not* have that environment variable set, nothing should be changed.
There's also a setting, per @clayborg 's suggestion, that will override
the environment variable, or can be used instead of the environment
variable. The setting is why I also needed to add an API to the
llvm-debuginfod library
### Test Plan:
Suggestions are welcome here. I should probably have some positive and
negative tests, but I wanted to get the diff up for people who have a
clue what they're doing to rip it to pieces before spending too much
time validating the initial implementation.
---------
Co-authored-by: Kevin Frei <freik@meta.com>
Co-authored-by: Alex Langford <nirvashtzero@gmail.com>
|
|
Many sources show that xxh3 is much better than xxh64. This particular
instance may or may not have noticeable difference, but this change
moves us toward removing xxHash64.
This change will invalid cache.
Reviewed By: mysterymath
Differential Revision: https://reviews.llvm.org/D155993
|
|
This makes parsing for build IDs in the markup filter slightly more
permissive, in line with fromHex.
It also removes the distinction between missing build ID and empty build
ID; empty build IDs aren't a useful concept, since their purpose is to
uniquely identify a binary. This removes a layer of indirection wherever
build IDs are obtained.
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D147485
|
|
Previously, the size of the debuginfod cache would grow without bound.
This change prunes the cache after a successful debuginfod lookup, as is
done in libdebuginfod.
The cache pruning behavior is configured by a new
DEBUGINFOD_CACHE_POLICY environment variable. The semantics of this are
the same as --thinlto_cache_policy.
Reviewed By: gulfem
Differential Revision: https://reviews.llvm.org/D145618
|
|
Reviewed By: gulfem
Differential Revision: https://reviews.llvm.org/D136702
|
|
This reverts commit efbc8bb18eda63007216ad0cb5a8de04963eddd5.
This change is causing failures when detecting curl on several build bots:
- https://lab.llvm.org/buildbot/#/builders/247/builds/884
- https://lab.llvm.org/buildbot/#/builders/231/builds/7688
- https://lab.llvm.org/buildbot/#/builders/121/builds/27389
- https://lab.llvm.org/buildbot/#/builders/230/builds/8464
- https://lab.llvm.org/buildbot/#/builders/57/builds/24209
- https://lab.llvm.org/buildbot/#/builders/127/builds/42722
|
|
Reviewed By: gulfem
Differential Revision: https://reviews.llvm.org/D136702
|
|
This is part of an effort to migrate from llvm::Optional to
std::optional:
https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
|
|
std::optional::value() has undesired exception checking semantics and is
unavailable in some older Xcode. The call sites block std::optional migration.
|
|
value() has undesired exception checking semantics and calls
__throw_bad_optional_access in libc++. Moreover, the API is unavailable without
_LIBCPP_NO_EXCEPTIONS on older Mach-O platforms (see
_LIBCPP_AVAILABILITY_BAD_OPTIONAL_ACCESS).
This fixes check-llvm.
|
|
|
|
This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.
This is part of an effort to migrate from llvm::Optional to
std::optional:
https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
|
|
This reverts commit 34108082947c964ae9bbfcd9808f2fd31c0d672f with fixes.
|
|
names.""
Breaks build of LLVMgold here:
```
/repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1108:19: error: no matching function for call to 'localCache'
Cache = check(localCache("ThinLTO", "Thin", options::cache_dir, AddBuffer));
^~~~~~~~~~
/repositories/llvm-project/llvm/include/llvm/Support/Caching.h:72:21: note: candidate function not viable: no known conversion from '(lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1102:20)' to 'llvm::AddBufferFn' (aka 'function<void (unsigned int, const llvm::Twine &, std::unique_ptr<MemoryBuffer>)>') for 4th argument
Expected<FileCache> localCache(
^
/repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1110:18: error: no viable conversion from '(lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20)' to 'llvm::AddStreamFn' (aka 'function<Expected<std::unique_ptr<CachedFileStream>> (unsigned int, const llvm::Twine &)>')
check(Lto->run(AddStream, Cache));
^~~~~~~~~
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/std_function.h:375:7: note: candidate constructor not viable: no known conversion from '(lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20)' to 'std::nullptr_t' for 1st argument
function(nullptr_t) noexcept
^
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/std_function.h:386:7: note: candidate constructor not viable: no known conversion from '(lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20)' to 'const std::function<llvm::Expected<std::unique_ptr<llvm::CachedFileStream>> (unsigned int, const llvm::Twine &)> &' for 1st argument
function(const function& __x)
^
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/std_function.h:404:7: note: candidate constructor not viable: no known conversion from '(lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20)' to 'std::function<llvm::Expected<std::unique_ptr<llvm::CachedFileStream>> (unsigned int, const llvm::Twine &)> &&' for 1st argument
function(function&& __x) noexcept
^
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/std_function.h:435:2: note: candidate template ignored: requirement '_Callable<(lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20) &, (lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20), std::__invoke_result<(lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20) &, unsigned int, const llvm::Twine &>>::value' was not satisfied [with _Functor = (lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20) &]
function(_Functor&& __f)
^
/repositories/llvm-project/llvm/include/llvm/LTO/LTO.h:278:25: note: passing argument to parameter 'AddStream' here
Error run(AddStreamFn AddStream, FileCache Cache = nullptr);
^
```
This reverts commit 387620aa8cea33174b6c1fb80c1af713fee732ac.
|
|
This reverts commit eef5405f74ae208e3e2eb7daacecac923d7338f2.
|
|
This reverts commit 531ed6d5aa65f41c6dfe2e74905d5c6c88fc95a7.
|
|
Currently the lto native object files have names like main.exe.lto.1.obj. In
PDB, those names are used as names for each compiland. Microsoft’s tool
SizeBench uses those names to present to users the size of each object files.
So, names like main.exe.lto.1.obj is not user friendly.
This patch makes the lto native object file names more readable by using
the bitcode file names as part of the file names. For example, if the input
bitcode file has path like "path/to/foo.obj", its corresponding lto native
object file path would be "path/to/main.exe.lto.foo.obj". Since the lto native
object file name only bothers PDB, this patch only changes the lld-linker's
behavior.
Reviewed By: tejohnson, MaskRay, #lld-macho
Differential Revision: https://reviews.llvm.org/D137217
|
|
|
|
This change adds a DEBUGINFOD_HEADERS_FILE environment variable provides
a file containing HTTP headers to attach to outgoing HTTP requests, one
per line. This allows a file permissioned with OS access control
mechanisms to supply bearer credentials for Debuginfod requests.
This matches the mechanism recently added to elfutils' libdebuginfod.
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D136303
|
|
This creates a library for fetching debug info by build ID, whether
locally or remotely via debuginfod. The functionality was refactored
out of existing code in the Symboliize library. Existing utilities
were refactored to use this library.
Reviewed By: phosek
Differential Revision: https://reviews.llvm.org/D132504
|
|
One of our silent bots is currently failing:
https://lab.llvm.org/staging/#/builders/171/builds/169
With:
<...>/Debuginfod.cpp:298:23: error: no type named 'sleep_for' in namespace 'std::this_thread'
std::this_thread::sleep_for(Interval);
~~~~~~~~~~~~~~~~~~^
Add missing thread include to that file,
which is what all the other users of sleep_for do.
I think we are seeing this now because we disabled
llvm threading for this builder. Maybe debuginfod should account
for that but that's for another time.
|
|
|
|
local debuginfo.
This library implements the class `DebuginfodCollection`, which scans a set of directories for binaries, classifying them according to whether they contain debuginfo. This also provides the `DebuginfodServer`, an `HTTPServer` which serves debuginfod's `/debuginfo` and `/executable` endpoints. This is intended as the final new supporting library required for `llvm-debuginfod`.
As implemented here, `DebuginfodCollection` only finds ELF binaries and DWARF debuginfo. All other files are ignored. However, the class interface is format-agnostic. Generalizing to support other platforms will require refactoring of LLVM's object parsing libraries to eliminate use of `report_fatal_error` ([[ https://github.com/llvm/llvm-project/blob/main/llvm/lib/Object/WasmObjectFile.cpp#L74 | e.g. when reading WASM files ]]), so that the debuginfod daemon does not crash when it encounters a malformed file on the disk.
The `DebuginfodCollection` is tested by end-to-end tests of the debuginfod server (D114846).
Reviewed By: mysterymath
Differential Revision: https://reviews.llvm.org/D114845
|
|
|
|
The present implementation of debuginfod lookups requires the
Content-Length field to be populated in the HTTP server response.
Unfortunately, Content-Length is optional, and there are some real
scenarios where it's missing. (For example, a Google Cloud Storage
server doing on-the-fly gunzipping.)
This changes the debuginfod response handler to directly stream the
output to the cache file as it is received. In addition to allowing
lookups to proceed without a Content-Lenght, it seems somewhat more
straightforward to implement, and it allows the disk I/O to be
interleaved with the network I/O.
Reviewed By: noajshu
Differential Revision: https://reviews.llvm.org/D121720
|
|
The cleanup was manual, but assisted by "include-what-you-use". It consists in
1. Removing unused forward declaration. No impact expected.
2. Removing unused headers in .cpp files. No impact expected.
3. Removing unused headers in .h files. This removes implicit dependencies and
is generally considered a good thing, but this may break downstream builds.
I've updated llvm, clang, lld, lldb and mlir deps, and included a list of the
modification in the second part of the commit.
4. Replacing header inclusion by forward declaration. This has the same impact
as 3.
Notable changes:
- llvm/Support/TargetParser.h no longer includes llvm/Support/AArch64TargetParser.h nor llvm/Support/ARMTargetParser.h
- llvm/Support/TypeSize.h no longer includes llvm/Support/WithColor.h
- llvm/Support/YAMLTraits.h no longer includes llvm/Support/Regex.h
- llvm/ADT/SmallVector.h no longer includes llvm/Support/MemAlloc.h nor llvm/Support/ErrorHandling.h
You may need to add some of these headers in your compilation units, if needs be.
As an hint to the impact of the cleanup, running
clang++ -E -Iinclude -I../llvm/include ../llvm/lib/Support/*.cpp -std=c++14 -fno-rtti -fno-exceptions | wc -l
before: 8000919 lines
after: 7917500 lines
Reduced dependencies also helps incremental rebuilds and is more ccache
friendly, something not shown by the above metric :-)
Discourse thread on the topic: https://llvm.discourse.group/t/include-what-you-use-include-cleanup/5831
|
|
This prevents the debuginfod client from dumping files directly into the
default cache directory (e.g., ~/.cache). Instead, these files are
placed in a subdirectory (e.g., ~/.cache/llvm-debuginfod/client).
Behavior is unaffected if the cache directory is provided by the
DEBUGINFO_CACHE_PATH environment variable.
Patch By: mysterymath
Differential Revision: https://reviews.llvm.org/D117619
|
|
Identified with modernize-use-nullptr.
|
|
Adds a fallback to use the debuginfod client library (386655) in `findDebugBinary`.
Fixed a cast of Erorr::success() to Expected<> in debuginfod library.
Added Debuginfod to Symbolize deps in gn.
Updates compiler-rt/lib/sanitizer_common/symbolizer/scripts/build_symbolizer.sh to include Debuginfod library to fix sanitizer-x86_64-linux breakage.
Reviewed By: jhenderson, vitalybuka
Differential Revision: https://reviews.llvm.org/D113717
|
|
This reverts commit 5bba0fe12b2971a9cbc859f48ee6e6c1356c88b8.
Makes lld depend on libcurl, see comments on https://reviews.llvm.org/D113717
|
|
Adds a fallback to use the debuginfod client library (386655) in `findDebugBinary`.
Fixed a cast of Erorr::success() to Expected<> in debuginfod library.
Added Debuginfod to Symbolize deps in gn.
Updates compiler-rt/lib/sanitizer_common/symbolizer/scripts/build_symbolizer.sh to include Debuginfod library to fix sanitizer-x86_64-linux breakage.
Reviewed By: jhenderson, vitalybuka
Differential Revision: https://reviews.llvm.org/D113717
|
|
This reverts commit e2ad4f1756027cd27f6c82db620042e9877f900c because it
does not correctly fix the sanitizer buildbot breakage.
|
|
Adds a fallback to use the debuginfod client library (386655) in `findDebugBinary`.
Fixed a cast of Erorr::success() to Expected<> in debuginfod library.
Added Debuginfod to Symbolize deps in gn.
Adds new symbolizer symbols to `global_symbols.txt`.
Reviewed By: jhenderson
Differential Revision: https://reviews.llvm.org/D113717
|
|
Following the discussion in D112753, this moves the HTTPClient from Support to Debuginfod library so that tools depending on Support do not automatically depend on Curl as well. This also removes `HTTPClient::initialize()` and `HTTPClient::cleanup()` from `InitLLVM` so these steps should be implemented by user tools instead.
Reviewed By: phosek
Differential Revision: https://reviews.llvm.org/D115131
|
|
This adds a Debuginfod client library which queries servers specified by the `DEBUGINFOD_URLS` environment variable for the debuginfo, executable, or a specified source file associated with a given build id.
Reviewed By: dblaikie
Differential Revision: https://reviews.llvm.org/D112758
|