Age | Commit message (Collapse) | Author | Files | Lines |
|
Most ORC APIs work with ExecutorAddr by default since 8b1771bd9f3, so we don't
need to wrap these values in ExecutorAddr(...) calls any more.
|
|
|
|
|
|
Tested this with the new AArch32 backend on armv7l and it works without issues in GDB. The size of the load-address field is only 32-bit here, but we implicitly account for it by writing a ELFT::uint which is:
https://github.com/llvm/llvm-project/blob/release/16.x/llvm/include/llvm/Object/ELFTypes.h#L57
So, instead of adding a newly supported machine type, let's just drop this restriction althogether.
|
|
Sometimes it's useful to be able and debug code even without actual debug info, e.g. for setting breakpoints on function names.
This patch adds a new API option to make it possible in Orc.
The existing API and behavior remains unchanged: non-debug objects are not passed to exectuors.
|
|
|
|
DebugObjectManagerPlugin.cpp:372:8: error: unused variable 'ItInserted' [-Werror,-Wunused-variable]
auto ItInserted = Sections.try_emplace(Name, std::move(Section));
^
|
|
When the initial DebugObjectManagerPlugin landed, it was not clear whether we will have more patching requirements for debug section. Also, there were no other use-cases for debug object flags.
Adding options to the plugin gives us a use-case and we can re-use the field for it. This commit only refactors the infrastructure in preparation for two more patches to come.
|
|
Originally, the DebugObjectManagerPlugin recorded all sections and filtered some of them for load-address patching.
Then we spotted problems with duplicate section names and started additional filtering upfront (see b26f45e5a49ae363164e7dbbf57eadd9e78d612c).
This seems the better approach. Let's go for it and stop filtering in two locations.
|
|
Compiler-generated section names can clash. Examples are group sections or profile counter sections.
We don't need to abort debug registration for the entire LinkGraph in such a case.
Instead, let's skip the relevant sections and add a note to the debug log.
|
|
|
|
ExecutorAddr was introduced in b8e5f918166 as an eventual replacement for
JITTargetAddress. ExecutorSymbolDef is introduced in this patch as a
replacement for JITEvaluatedSymbol: ExecutorSymbolDef is an (ExecutorAddr,
JITSymbolFlags) pair, where JITEvaluatedSymbol was a (JITTargetAddress,
JITSymbolFlags) pair.
A number of APIs had already migrated from JITTargetAddress to ExecutorAddr,
but many of ORC's internals were still using the older type. This patch aims
to address that.
Some public APIs are affected as well. If you need to migrate your APIs you can
use the following operations:
* ExecutorAddr::toPtr replaces jitTargetAddressToPointer and
jitTargetAddressToFunction.
* ExecutorAddr::fromPtr replace pointerToJITTargetAddress.
* ExecutorAddr(JITTargetAddress) creates an ExecutorAddr value from a
JITTargetAddress.
* ExecutorAddr::getValue() creates a JITTargetAddress value from an
ExecutorAddr.
JITTargetAddress and JITEvaluatedSymbol will remain in JITSymbol.h for now, but
the aim will be to eventually deprecate and remove these types (probably when
MCJIT and RuntimeDyld are deprecated).
|
|
The LLVMOrcObjectLayerAddObjectFileWithRT method does not have an
implementation. LLVMOrcLLJITAddObjectFileWithRT exists twice (maybe a copy
paste error).
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D118447
|
|
Where the same dylib is loaded more than once we should just return the
JITDylib created by the first call rather than error out. This matches the
behavior of dlopen / LoadLibrary.
|
|
LLJIT::loadPlatformDynamicLibrary loads a dynamic library at a given path
(interpreted in the executor process -- the process containing the JIT'd code),
and returns a JITDylib (whose name is the given path) that reflects the symbols
in that library. LLJIT clients wishing to make the given symbols visible to
their JIT'd code can add this JITDylib to the link order of their JITDylib(s)
using JITDylib::addToLinkOrder.
The LLJIT::linkStaticLibraryInto overloads load a static library (or universal
binary) at a given path (interpreted in the controller process -- the process
containing the LLJIT instance) and adds its symbols to the given JITDylib.
The lli tool is updated to use LLJIT::linkStaticLibraryInto to implement the
extra-archive option.
LLJIT::loadPlatformDynamicLibrary is not tested in this patch as we don't have
a good way to produce dylibs in LLVM's regression test suite.
|
|
This first version lays the foundations for AArch32 support in JITLink. ELFLinkGraphBuilder_aarch32 processes REL-type relocations and populates LinkGraphs from ELF object files for both big- and little-endian systems. The ArmCfg member controls subarchitecture-specific details throughout the linking process (i.e. it's passed to ELFJITLinker_aarch32).
Relocation types follow the ABI documentation's division into classes: Data (endian-sensitive), Arm (32-bit little-endian) and Thumb (2x 16-bit little-endian, "Thumb32" in the docs). The implementation of instruction encoding/decoding for relocation resolution is implemented symmetrically and is testable in isolation (see AArch32 category in JITLinkTests).
Callable Thumb functions are marked with a ThumbSymbol target-flag and stored in the LinkGraph with their real addresses. The thumb-bit is added back in when the owning JITDylib requests the address for such a symbol.
The StubsManager can generate (absolute) Thumb-state stubs for branch range extensions on v7+ targets. Proper GOT/PLT handling is not yet implemented.
This patch is based on the backend implementation in ez-clang and has just enough functionality to model the infrastructure and link a Thumb function `main()` that calls `printf()` to dump "Hello Arm!" on Armv7a. It was tested on Raspberry Pi with 32-bit Raspbian OS.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D144083
|
|
This reverts commit c2de8ff92753acdb1ace7a27cc11cb09f28eb8fa.
It caused a segmentation fault while running ExecutionEngine
tests on Mac.
https://luci-milo.appspot.com/ui/p/fuchsia/builders/toolchain.ci/clang-mac-x64/b8785839382041226465/overview
|
|
This first version lays the foundations for AArch32 support in JITLink. ELFLinkGraphBuilder_aarch32 processes REL-type relocations and populates LinkGraphs from ELF object files for both big- and little-endian systems. The ArmCfg member controls subarchitecture-specific details throughout the linking process (i.e. it's passed to ELFJITLinker_aarch32).
Relocation types follow the ABI documentation's division into classes: Data (endian-sensitive), Arm (32-bit little-endian) and Thumb (2x 16-bit little-endian, "Thumb32" in the docs). The implementation of instruction encoding/decoding for relocation resolution is implemented symmetrically and is testable in isolation (see AArch32 category in JITLinkTests).
Callable Thumb functions are marked with a ThumbSymbol target-flag and stored in the LinkGraph with their real addresses. The thumb-bit is added back in when the owning JITDylib requests the address for such a symbol.
The StubsManager can generate (absolute) Thumb-state stubs for branch range extensions on v7+ targets. Proper GOT/PLT handling is not yet implemented.
This patch is based on the backend implementation in ez-clang and has just enough functionality to model the infrastructure and link a Thumb function `main()` that calls `printf()` to dump "Hello Arm!" on Armv7a. It was tested on Raspberry Pi with 32-bit Raspbian OS.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D144083
|
|
Sort symbols before dumping so we get a deterministic order and can check them in tests.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D146658
|
|
to account for them
AArch32 branch offsets explicitly encode the target instruction subset (Arm/Thumb) in their least significant bit. We want this bit set (or clear) in addreses we hand out, but the addresses in the LinkGraph should be the real/physical addresses.
This patch allows ELFLinkGraphBuilder's to set target-specific flags in jitlink::Symbol and prepares ObjectLinkingLayer to account for them.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D146641
|
|
Update JITDylib's symbol table entry struct to use the newer ExecutorAddr type.
|
|
This reverts commit bdf5f9c3228d6ed1d7c6f87b3828a7d573b34c03, which was a work
in progress for https://reviews.llvm.org/D144276 and accidentally committed
early.
|
|
Simplifies the process of building an LLJIT instance that supports the native
platform features (initializers, TLV, etc.).
SetUpExecutorNativePlatform can be passed to LLJITBuilder::setPlatformSetUp
method. It takes a reference to the ORC runtime (as a path or an in-memory
archive) and automatically sets the platform for LLJIT's ExecutionSession based
on the executor process's triple.
Differential Revision: https://reviews.llvm.org/D144276
|
|
This reapplies 2cc64df0bd6a802eab592dbc282463c3e4a4281c, which was reverted in
5379c46d490640bfa80283e00240b6f1006092b4 due to bot failures.
The new patch contains fixes to ELFLinkGraphBuilder.h to better handle
non-SHT_ALLOC sections (these were being accidentally skipped in the previous
patch), and to skip SHT_NULL sections.
|
|
This reverts commit 2cc64df0bd6a802eab592dbc282463c3e4a4281c while I investigate
bot failures (e.g. https://lab.llvm.org/buildbot/#/builders/3/builds/23081).
|
|
The original MemDeallocPolicy had two options:
* Standard: allocated memory lives until deallocated or abandoned.
* Finalize: allocated memory lives until all finalize actions have been run,
then is destroyed.
This patch introduces a new 'NoAlloc' option. NoAlloc indicates that the
section should be ignored by the JITLinkMemoryManager -- the memory manager
should allocate neither working memory nor executor address space to blocks in
NoAlloc sections. The NoAlloc option is intended to support metadata sections
(e.g. debug info) that we want to keep in the graph and have fixed up if
necessary, but don't want allocated or transmitted to the executor (or we want
that allocation and transmission to be managed manually by plugins).
Since NoAlloc blocks are ignored by the JITLinkMemoryManager they will not have
working memory allocated to them by default post-allocation. Clients wishing to
modify the content of a block in a NoAlloc section should call
`Block::getMutableMemory(LinkGraph&)` to get writable memory allocated on the
LinkGraph's allocator (this memory will exist for the lifetime of the graph).
If no client requests mutable memory prior to the fixup phase then the generic
link algorithm will do so when it encounters the first edge in any given block.
Addresses of blocks in NoAlloc sections are initialized by the LinkGraph
creator (a LinkGraphBuilder, if the graph is generated from an object file),
and should not be modified by the JITLinkMemoryManager. Plugins are responsible
for updating addresses if they add/remove content from these sections. The
meaning of addresses in NoAlloc-sections is backend/plugin defined, but for
fixup purposes they will be treated the same as addresses in Standard/Finalize
sections. References from Standard/Finalize sections to NoAlloc sections are
expected to be common (these represent metadata tracking executor addresses).
References from NoAlloc sections to Standard/Finalize sections are expected to
be rare/non-existent (they would represent JIT'd code / data tracking metadata
in the controller, which would be surprising). LinkGraphBuilders and specific
backends may impose additional constraints on edges between Standard/Finalize
and NoAlloc sections where required for correctness.
Differential Revision: https://reviews.llvm.org/D146183
|
|
Extend the logic for the WinSDK and UCRT handling to prefer a user
specified version of the VisualC++ tools and Windows SDK. This allows
us to now perform the regular search for the installation but select the
exact version of the SDK or VC++ tools to override the latest version.
Similar to the other flags controlling this behaviour, if the user
specifies a value, we will not perform validation on the input and will
attempt to prefer that, particularly in the case of VisualC++ tools
where no fallback occurs.
Reviewed by: hans
Differential Revision: https://reviews.llvm.org/D145517
|
|
Testing the COFFPlatform on MSVC, a name shadowing issue surfaced where `LoadDynLibrary` inside the constructor was actually using the moved-from function argument. This patch simply renames the argument to avoid that shadowing.
Reviewed By: rriddle
Differential Revision: https://reviews.llvm.org/D145710
|
|
We can get the triple from the ExecutionSession, so clients shouldn't have to
provide it.
|
|
Forwards to ExecutorProcessControl::getTargetTriple, and saves clients the
trouble of spelling 'getExecutorProcessControl()' everywhere.
|
|
The existing Create method took a path to the ORC runtime and created a
StaticLibraryDefinitionGenerator for it. The new overload takes a
std::unique_ptr<DefinitionGenerator> directly instead. This provides more
flexibility when constructing MachOPlatforms. E.g. The runtime archive can be
embedded in a special section in the ORC controller executable or library,
rather than being on-disk.
This is the ELFNixPlatform equivalent of the MachOPlatform change in
be2fc577c38.
|
|
Add new StaticLibraryDefinitionGenerator::Create methods to support construction
from in-memory universal binaries.
|
|
The existing Create method took a path to the ORC runtime and created a
StaticLibraryDefinitionGenerator for it. The new overload takes a
std::unique_ptr<DefinitionGenerator> directly instead. This provides more
flexibility when constructing MachOPlatforms. E.g. The runtime archive can be
embedded in a special section in the ORC controller executable or library,
rather than being on-disk.
|
|
Renames the existing allocateString method to allocateContent and adds a pair of
allocateCString methods.
The previous allocateString method did not include a null-terminator. It behaved
the same as allocateContent except with a Twine input, rather than an
ArrayRef<char>. Renaming allocateString to allocateBuffer (overloading the
existing method) makes this clearer.
The new allocateCString methods allocate the given content plus a
null-terminator character, and return a buffer covering both the string and
null-terminator. This makes them suitable for creating c-string content for
jitlink::Blocks.
Existing users of the old allocateString method have been updated to use the
new allocateContent overload.
|
|
This allows these details to be shared with JITLink, which is allowed to
depend on the OrcShared library (but not on OrcJIT).
|
|
The forwarding header is left in place because of its use in
`polly/lib/External/isl/interface/extract_interface.cc`, but I have
added a GCC warning about the fact it is deprecated, because it is used
in `isl` from where it is included by Polly.
|
|
I also ran `git clang-format` to get the headers in the right order for
the new location, which has changed the order of other headers in two
files.
|
|
According to the IR verifier, "Declaration[s] may not be in a Comdat!"
This is a re-commit of 76b3f0b4d5a0b8c54147c4c73a30892bbca76467 and
87d7838202267a011639fcbf97263556ccf091dc with updates to the test:
* Force emission of the extra-module, to trigger the bug after D138264,
by providing a second symbol @g, and making the comdat nodeduplicate.
(Technically only one is needed, but two should be safer.)
* Name the comdat $f to avoid failure on Windows:
LLVM ERROR: Associative COMDAT symbol 'c' does not exist.
* Mark the test as UNSUPPORTED on macOS, MachO doesn't support COMDATs.
Differential Revision: https://reviews.llvm.org/D142443
|
|
Also updates the OrcCAPIsTest unit test to enable the C API tests on Linux.
|
|
A number of AArch64 bots report errors in clang-repl, for example
https://lab.llvm.org/buildbot/#/builders/197/builds/3920
This reverts commit 87d7838202267a011639fcbf97263556ccf091dc.
|
|
According to the IR verifier, "Declaration[s] may not be in a Comdat!"
This is a re-commit of 76b3f0b4d5a0b8c54147c4c73a30892bbca76467 with
updates to the test:
* Force emission of the extra-module, to trigger the bug after D138264,
by providing a second symbol @g, and making the comdat nodeduplicate.
(Technically only one is needed, but two should be safer.)
* Name the comdat $f to avoid failure on Windows:
LLVM ERROR: Associative COMDAT symbol 'c' does not exist.
Differential Revision: https://reviews.llvm.org/D142443
|
|
Failure on Windows:
LLVM ERROR: Associative COMDAT symbol 'c' does not exist.
This reverts commit 76b3f0b4d5a0b8c54147c4c73a30892bbca76467 while
I investigate the problem and a solution that still triggers the
original problem.
|
|
According to the IR verifier, "Declaration[s] may not be in a Comdat!"
Differential Revision: https://reviews.llvm.org/D142443
|
|
Adds a getJITSymbolFlagsForSymbol function that returns the JITSymbolFlags
for a given jitlink::Symbol, and replaces severalredundant copies of that
mapping with calls to the new function. This fixes a bug in
LinkGraphMaterializationUnit::scanLinkGraph where we were failing to set the
JITSymbolFlags::Weak flag for weak symbols, and a bug in
ObjectLinkingLayer::claimOrExternalizeWeakAndCommonSymbols where we were
failing to set the JITSymbolFlags::Callable flag for callable symbols.
|
|
The loop body may add and remove entries in the symbol table so we can't hold
iterators to the entries. This commit updates the method to use the newly added
NonOwningSymbolStringPtr type as keys for removal instead.
Side note: This bug has been present since the introduction of the
defineMaterializing method, but the method is called rarely and DenseMap
resizes are also rare so we didn't see any fallout until a large program was
thrown at it. There's no testcase as I haven't been able to reproduce the
failure with smaller testcases.
|
|
weak.
Patch by Lang Hames and Sunho Kim!
Differential revision: https://reviews.llvm.org/D138264
|
|
This patch adds resolver, indirection and trampoline stubs for
loongarch64, allowing lazy compilation to work.
It assumes hard float feature exists.
Depends on D141036
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D141102
|
|
In LLVM the MachOPlatform class is modified to identify unwind info sections
and the address ranges of the functions these sections cover. These address
ranges are then communicated to the ORC runtime by attaching them to the
register-object-platform-sections allocation action.
In the ORC runtime the unwind-info section addresses are recorded and used to
support lookup of unwind info via the new `findDynamicUnwindSections` function.
At bootstrap time the ORC runtime checks for the presence of new
unwind-info-lookup-registration functions in libunwind (see
https://reviews.llvm.org/D142176), and if available uses them to register the
`findDynamicUnwindSections` function with libunwind to enable callback-based
lookup. If the new unwind-info-lookup-registration functions are not available
then the ORC runtime falls back to using the existing libunwind registration
APIs.
The callback-based scheme is intended to address three shortcomings in the
current registration scheme for JIT'd unwind info on Darwin: (1) Lack of
compact-unwind support, (2) inability to describe the subarchitecture of JIT'd
frames, and (3) lack of efficient address-based lookup data structures in
libunwind.
For more details see the proposed libunwind changes in
https://reviews.llvm.org/D142176.
|
|
This implementation supports basic relocation types and adds EHFrame,
Got/Plt handling passes.
This patch also enables JIT support for LoongArch64.
With this patch, I successfully run hello.ll and simple_throw.ll
(which is generated from test-suite/SingleSource/Regression/C++/EH/simple_throw.cpp)
using the `lli` command with options `--jit-kind=orc --jit-linker=jitlink`.
Note: `hasJIT` property of LoongArch32 remains false as there is no
validation environment.
New changes: Since LoongArch does not support RuntimeDyld, JITLink is set
by default. Add a null-terminator to eh-frame sections. This should fix
the test failure on LoongArch bot.
(https://lab.llvm.org/staging/#/builders/236/builds/896)
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D141036
|
|
This patch modifies the MachOPlatform bootstrap process to record allocation
actions for ORC runtime platform support code in a "deferred actions" vector
rather than attaching it to the corresponding LinkGraphs up-front. The deferred
allocation-actions are run after all the platform support code has been loaded
by attaching them to a separate "bootstrap-complete" graph.
This change should allow the mach-o platform support code in the ORC runtime to
use advanced mach-o platform features (e.g. static inits, TLVs), provided that
the support code does not use these features at runtime before the bootstrap
process completes, or after the shutdown process starts. This is a nice
improvement in and of itself but is motivated by specific future plans: we
want to start recording unwind info in the mach-o platform state object*, and
the recording functions will have their own frame info that needs registering.
The deferred allocation-actions scheme allows for this.
* The plan is to add a new unwind-info-lookup path to libunwind to allow it to
call back to the ORC runtime to find unwind sections. This will simplify the
implementation of support for JIT'd compact-unwind info.
|