| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
This was accidentally introduced in an earlier commit.
|
|
I'm planning to deprecate and eventually remove llvm::empty.
I thought about replacing llvm::empty(x) with std::empty(x), but it
turns out that all uses can be converted to x.empty(). That is, no
use requires the ability of std::empty to accept C arrays and
std::initializer_list.
Differential Revision: https://reviews.llvm.org/D133677
|
|
Removes public bootstrap method that is not really necessary and not consistent with other platform API.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D132780
|
|
Supports dynamic VC runtime. It implements atexits handling which is required to load msvcrt.lib successfully. (the object file containing atexit symbol somehow resolves to static vc runtim symbols) It also default to dynamic vc runtime which tends to be more robust.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D132525
|
|
tool
MapperJITLinkMemoryManager combined with SharedMemoryMapper can be
used in place of EPCGenericJITLinkMemoryManager when running on top of
same physical memory. This commit enables it when --use-shared-memory is
passed.
Differential Revision: https://reviews.llvm.org/D132369
|
|
MapperJITLinkMemoryManager has slab allocation. Combined with
InProcessMapper, it can replace InProcessMemoryManager.
It can also replace JITLinkSlabAllocator through the InProcessDeltaMapper
that adds an offset to the executor addresses for use in tests.
Differential Revision: https://reviews.llvm.org/D132315
|
|
Initial platform support for COFF/x86_64.
Completed features:
* Statically linked orc runtime.
* Full linking/initialization of static/dynamic vc runtimes and microsoft stl libraries.
* SEH exception handling.
* Full static initializers support
* dlfns
* JIT side symbol lookup/dispatch
Things to note:
* It uses vc runtime libraries found in vc toolchain installations.
* Bootstrapping state is separated because when statically linking orc runtime it needs microsoft stl functions to initialize the orc runtime, but static initializers need to be ran in order to fully initialize stl libraries.
* Process symbols can't be used blidnly on msvc platform; otherwise duplicate definition error gets generated. If process symbols are used, it's destined to get out-of-reach error at some point.
* Atexit currently not handled -- will be handled in the follow-up patches.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D130479
|
|
Add a reference to llvm_orc_registerJITLoaderGDBAllocAction from the
linkComponents function in the lli, llvm-jitlink, and llvm-jitlink-executor
tools. This ensures that llvm_orc_registerJITLoaderGDBAllocAction is not
dead-stripped in optimized builds, which may cause failures in these tools.
The llvm_orc_registerJITLoaderGDBAllocAction function was originally added with
MachO debugging support in 69be352a1961a, but that patch failed to update the
linkComponents functions.
http://llvm.org/PR56817
|
|
Handles COFF import files of static archive. Changes static library genrator to build up object file map keyed by symbol name that excludes the symbols from dllimported symbols so that static generator will not be responsible for them. It exposes the list of dynamic libraries that need to be imported. Client should properly load the libraries in this list beforehand. Object file map is also an improvment from the past in terms of performance. Archive.findSym does a slow O(n) linear serach of symbol list to find the symbol. (we call findSym O(n) times, thus full time complexity is O(n^2); we were the only user of findSym function in fact)
There is a room for improvements in how to load the libraries in the list. We currently just hand the responsibility over to the clinet. A better way would be let ORC read this list and hand them over to JITLink side that would also help validation (e.g. not trying to generate stub for non dllimported targets) Nevertheless, we will have to exclude the symbols from COFF import object file list and need a way to access this list, which this patch offers.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D129952
|
|
Consider lib/dll files in llvm-jitlink.
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D129944
|
|
Adds initial COFF support in JITLink. This is able to run a hello world c program in x86 windows successfully.
Implemented
- COFF object loader
- Static local symbols
- Absolute symbols
- External symbols
- Weak external symbols
- Common symbols
- COFF jitlink-check support
- All COMDAT selection type execpt largest
- Implicit symobl size calculation
- Rel32 relocation with PLT stub.
- IMAGE_REL_AMD64_ADDR32NB relocation
Reviewed By: lhames
Differential Revision: https://reviews.llvm.org/D128968
|
|
|
|
|
|
|
|
|
|
This patch makes printing of FailedToMaterialize errors in llvm-jitlink
conditional on the -show-err-failed-to-materialize option, which defaults to
false.
FailedToMaterialize errors are not root-cause errors: they're generated when a
symbol is requested but cannot be provided because of a failure that was
reported on some other error path. They typically don't convey actionable
information, and tend to flood error logs making root cause errors harder to
spot. Hiding FailedToMaterialize errors by default addresses these issues.
|
|
The directive pollutes the namespace of all files which include the
header.
Use alternate ways to reference the namespace constituents instead.
|
|
The slab delta (used to link as if allocated at a specified address) should
remain constant.The update to the delta was accidentally introduced in
962a2479b57f5, but hasn't caused any failures as it only breaks in an obvious
way for multi-file exec uses (our regression tests are all -noexec, and tend to
be single-file).
No testcase here: this is an obscure utility for testing support, and an
uncommon use-case. If the slab allocator is ever moved into LLVM we could add
a unit test to catch this.
|
|
The addProcessSymbols function added a generator for process symbols to every
JITDylib in the session, but this is unhelpful default behavior (e.g. it will
cause the ORC runtime's definition of __cxa_atexit to be shadowed by the
process's definition for all JITDylibs except main).
Since the loadProcessSymbols function already added a generator to main we only
need to drop this function. Other JITDylibs wishing to resolve process symbols
can link against the main JITDylib by passing `-lmain`.
|
|
The -alias option can be used to define aliases within a JITDylib. The
immediate motivation is to simplify testing of ORC runtime functions using
existing testcases (e.g. by aliasing dlfcn functions to their ORC-runtime
counterparts, like -alias dlopen=__orc_rt_macho_dlopen). The option is likely
to be useful for testing in general.
The -define-abs option is shortened to -abs for consistency with -alias.
|
|
`llvm_jitlink_setTestResultOverride` is used via runtime lookup by
tests, so make sure it is not dead-stripped from llvm-jitlink in release
builds.
Fixes https://github.com/llvm/llvm-project/issues/53203
Differential Revision: https://reviews.llvm.org/D117609
|
|
Similar to the ld64 command-line options. These use the same underlying
mechanisms as -l and -hidden-l, but allow specifying an absolute path to
the archive. This is often more convenient for a one-off, or when adding
a new search path could change how existing -l options are resolved.
Differential Revision: https://reviews.llvm.org/D117360
|
|
runFinalizeActions takes an AllocActions vector and attempts to run its finalize
actions. If any finalize action fails then all paired dealloc actions up to the
failing pair are run, and the error(s) returned. If all finalize actions succeed
then a vector containing the dealloc actions is returned.
runDeallocActions takes a vector<WrapperFunctionCall> containing dealloc action
calls and runs them all, returning any error(s).
These helpers are intended to simplify the implementation of
JITLinkMemoryManager::InFlightAlloc::finalize and
JITLinkMemoryManager::deallocate overrides by taking care of execution (and
potential roll-back) of allocation actions.
|
|
These types performed identical roles. Merging them simplifies interoperability
between JITLink and ORC APIs (allowing us to address a few FIXMEs).
|
|
This re-applies 133f86e95492b2a00b944e070878424cfa73f87c, which was reverted in
c5965a411c635106a47738b8d2e24db822b7416f while I investigated bot failures.
The original failure contained an arithmetic conversion think-o (on line 419 of
EHFrameSupport.cpp) that could cause failures on 32-bit platforms. The issue
should be fixed in this patch.
|
|
This reverts commit 133f86e95492b2a00b944e070878424cfa73f87c while I investigate
the bot failures at https://lab.llvm.org/buildbot#builders/186/builds/3370.
|
|
ExecutorAddr is the preferred representation for executor process addresses now.
|
|
Identified with modernize-use-nullptr.
|
|
This adds a GetObjectFileInterface callback member to
StaticLibraryDefinitionGenerator, and adds an optional argument for initializing
that member to StaticLibraryDefinitionGenerator's named constructors. If not
supplied, it will default to getObjectFileInterface from ObjectFileInterface.h.
To enable testing a `-hidden-l<x>` option is added to the llvm-jitlink tool.
This allows archives to be loaded with all contained symbol visibilities demoted
to hidden.
The ObjectLinkingLayer::setOverrideObjectFlagsWithResponsibilityFlags method is
(belatedly) hooked up, and enabled in llvm-jitlink when `-hidden-l<x>` is used
so that the demotion is also applied at symbol resolution time (avoiding any
"mismatched symbol flags" crashes).
|
|
|
|
Adds -L<search-path> and -l<library> options that are analogous to ld's
versions.
Each instance of -L<search-path> or -l<library> will apply to the most recent
-jd option on the command line (-jd <name> creates a JITDylib with the given
name). Library names will match against JITDylibs first, then llvm-jitlink will
look through the search paths for files named <search-path>/lib<library>.dylib
or <search-path>/lib<library>.a.
The default "main" JITDylib will link against all JITDylibs created by -jd
options, and all JITDylibs will link against the process symbols (unless
-no-process-symbols is specified).
The -dlopen option is renamed -preload, and will load dylibs into the JITDylib
for the ORC runtime only.
The effect of these changes is to make it easier to describe a non-trivial
program layout to llvm-jitlink for testing purposes. E.g. the following
invocation describes a program consisting of three JITDylibs: "main" (created
implicitly) containing main.o, "Foo" containing foo1.o and foo2.o, and linking
against library "bar" (not a JITDylib, so it must be a .dylib or .a on disk)
and "Baz" (which is a JITDylib), and "Baz" containing baz.o.
llvm-jitlink \
main.o \
-jd Foo foo1.o foo2.o -L${HOME}/lib -lbar -lBaz
-jd Baz baz.o
|
|
Also moves object interface building functions out of Mangling.h and in to the
new ObjectFileInterfaces.h header, and updates the llvm-jitlink tool to use
custom object interfaces rather than a custom link layer.
ObjectLayer::add overloads are added to match the old signatures (which
do not take a MaterializationUnit::Interface). These overloads use the
standard getObjectFileInterface function to build an interface.
Passing a MaterializationUnit::Interface explicitly makes it easier to alter
the effective interface of the object file being added, e.g. by changing symbol
visibility/linkage, or renaming symbols (in both cases the changes will need to
be mirrored by a JITLink pass at link time to update the LinkGraph to match the
explicit interface). Altering interfaces in this way can be useful when lazily
compiling (e.g. for renaming function bodies) or emulating linker options (e.g.
demoting all symbols to hidden visibility to emulate -load_hidden).
|
|
MaterializationUnit::Interface holds the values that make up the interface
(for ORC's purposes) of a materialization unit: the symbol flags map and
initializer symbol.
Having a type for this will make functions that build materializer interfaces
more readable and maintainable.
|
|
This is useful when debugging failures in object files compiled with
visibility=hidden.
|
|
Commit 69be352a196 restricted the MachO debugger support testcase to run on
Darwin only, but we still need to disable debugger support by default for
other noexec tests.
This patch introduces a -debugger-support option to llvm-jitlink that is
on-by-default when executing code, and off-by-default for noexec tests. This
should prevent regression tests from trying (and failing) to set up MachO
debugging support when running on non-Darwin platforms.
to explicitly enable/disable support.
|
|
This reapplies e1933a0488a50eb939210808fc895d374570d891 (which was reverted in
f55ba3525eb19baed7d3f23638cbbd880246a370 due to bot failures, e.g.
https://lab.llvm.org/buildbot/#/builders/117/builds/2768).
The bot failures were due to a missing symbol error: We use the input object's
mangling to decide how to mangle the debug-info registration function name. This
caused lookup of the registration function to fail when the input object
mangling didn't match the host mangling.
Disbaling the test on non-Darwin platforms is the easiest short-term solution.
I have filed https://llvm.org/PR52503 with a proposed longer term solution.
|
|
This reverts commit e1933a0488a50eb939210808fc895d374570d891 until I can look
into bot failures.
|
|
This commit adds a new plugin, GDBJITDebugInfoRegistrationPlugin, that checks
for objects containing debug info and registers any debug info found via the
GDB JIT registration API.
To enable this registration without redundantly representing non-debug sections
this plugin synthesizes a new embedded object within a section of the LinkGraph.
An allocation action is used to make the registration call.
Currently MachO only. ELF users can still use the DebugObjectManagerPlugin. The
two are likely to be merged in the near future.
|
|
The check was failing because it was matching against the end of the range, not
the start.
This bug wasn't causing the ORC-RT MachO TLV regression test to fail because
we were only logging deallocation errors (including TLV deregistration errors)
and not actually returning a failure code. This commit updates llvm-jitlink to
report the errors properly.
|
|
If a tool wants to introduce new indirections via stubs at link-time in
ORC, it can cause fidelity issues around the address of the function if
some references to the function do not have relocations. This is known
to happen inside the body of the function itself on x86_64 for example,
where a PC-relative address is formed, but without a relocation.
```
_foo:
leaq -7(%rip), %rax ## form pointer to '_foo' without relocation
_bar:
leaq (%rip), %rax ## uses X86_64_RELOC_SIGNED to '_foo'
```
The consequence of introducing a stub for such a function at link time
is that if it forms a pointer to itself without relocation, it will not
have the same value as a pointer from outside the function. If the
function pointer is used as a key, this can cause problems.
This utility provides best-effort support for adding such missing
relocations using MCDisassembler and MCInstrAnalysis to identify the
problematic instructions. Currently it is only implemented for x86_64.
Note: the related issue with call/jump instructions is not handled
here, only forming function pointers.
rdar://83514317
Differential revision: https://reviews.llvm.org/D113038
|
|
SimpleRemoteEPC notionally allowed subclasses to override the
createMemoryManager and createMemoryAccess methods to use custom objects, but
could not actually be subclassed in practice (The construction process in
SimpleRemoteEPC::Create could not be re-used).
Instead of subclassing, this commit adds a SimpleRemoteEPC::Setup class that
can be used by clients to set up the memory manager and memory access members.
A default-constructed Setup object results in no change from previous behavior
(EPCGeneric* memory manager and memory access objects used by default).
|
|
This should fix compile errors in llvm-jitlink.cpp in LLVM_ENABLE_THREADS=Off
builds due to f3411616896.
|
|
Adds explicit narrowing casts to JITLinkMemoryManager.cpp.
Honors -slab-address option in llvm-jitlink.cpp, which was accidentally
dropped in the refactor.
This effectively reverts commit 6641d29b70993bce6dbd7e0e0f1040753d38842f.
|
|
This warning should only be issued if -slab-page-size has not been used.
|
|
This reverts commit e50aea58d59c8cfae807a7fee21c4227472c0678 while I
investigate bot failures.
|
|
This commit substantially refactors the JITLinkMemoryManager API to: (1) add
asynchronous versions of key operations, (2) give memory manager implementations
full control over link graph address layout, (3) enable more efficient tracking
of allocated memory, and (4) support "allocation actions" and finalize-lifetime
memory.
Together these changes provide a more usable API, and enable more powerful and
efficient memory manager implementations.
To support these changes the JITLinkMemoryManager::Allocation inner class has
been split into two new classes: InFlightAllocation, and FinalizedAllocation.
The allocate method returns an InFlightAllocation that tracks memory (both
working and executor memory) prior to finalization. The finalize method returns
a FinalizedAllocation object, and the InFlightAllocation is discarded. Breaking
Allocation into InFlightAllocation and FinalizedAllocation allows
InFlightAllocation subclassses to be written more naturally, and FinalizedAlloc
to be implemented and used efficiently (see (3) below).
In addition to the memory manager changes this commit also introduces a new
MemProt type to represent memory protections (MemProt replaces use of
sys::Memory::ProtectionFlags in JITLink), and a new MemDeallocPolicy type that
can be used to indicate when a section should be deallocated (see (4) below).
Plugin/pass writers who were using sys::Memory::ProtectionFlags will have to
switch to MemProt -- this should be straightworward. Clients with out-of-tree
memory managers will need to update their implementations. Clients using
in-tree memory managers should mostly be able to ignore it.
Major features:
(1) More asynchrony:
The allocate and deallocate methods are now asynchronous by default, with
synchronous convenience wrappers supplied. The asynchronous versions allow
clients (including JITLink) to request and deallocate memory without blocking.
(2) Improved control over graph address layout:
Instead of a SegmentRequestMap, JITLinkMemoryManager::allocate now takes a
reference to the LinkGraph to be allocated. The memory manager is responsible
for calculating the memory requirements for the graph, and laying out the graph
(setting working and executor memory addresses) within the allocated memory.
This gives memory managers full control over JIT'd memory layout. For clients
that don't need or want this degree of control the new "BasicLayout" utility can
be used to get a segment-based view of the graph, similar to the one provided by
SegmentRequestMap. Once segment addresses are assigned the BasicLayout::apply
method can be used to automatically lay out the graph.
(3) Efficient tracking of allocated memory.
The FinalizedAlloc type is a wrapper for an ExecutorAddr and requires only
64-bits to store in the controller. The meaning of the address held by the
FinalizedAlloc is left up to the memory manager implementation, but the
FinalizedAlloc type enforces a requirement that deallocate be called on any
non-default values prior to destruction. The deallocate method takes a
vector<FinalizedAlloc>, allowing for bulk deallocation of many allocations in a
single call.
Memory manager implementations will typically store the address of some
allocation metadata in the executor in the FinalizedAlloc, as holding this
metadata in the executor is often cheaper and may allow for clean deallocation
even in failure cases where the connection with the controller is lost.
(4) Support for "allocation actions" and finalize-lifetime memory.
Allocation actions are pairs (finalize_act, deallocate_act) of JITTargetAddress
triples (fn, arg_buffer_addr, arg_buffer_size), that can be attached to a
finalize request. At finalization time, after memory protections have been
applied, each of the "finalize_act" elements will be called in order (skipping
any elements whose fn value is zero) as
((char*(*)(const char *, size_t))fn)((const char *)arg_buffer_addr,
(size_t)arg_buffer_size);
At deallocation time the deallocate elements will be run in reverse order (again
skipping any elements where fn is zero).
The returned char * should be null to indicate success, or a non-null
heap-allocated string error message to indicate failure.
These actions allow finalization and deallocation to be extended to include
operations like registering and deregistering eh-frames, TLS sections,
initializer and deinitializers, and language metadata sections. Previously these
operations required separate callWrapper invocations. Compared to callWrapper
invocations, actions require no extra IPC/RPC, reducing costs and eliminating
a potential source of errors.
Finalize lifetime memory can be used to support finalize actions: Sections with
finalize lifetime should be destroyed by memory managers immediately after
finalization actions have been run. Finalize memory can be used to support
finalize actions (e.g. with extra-metadata, or synthesized finalize actions)
without incurring permanent memory overhead.
|
|
ExecutorProcessControl objects will now have a TaskDispatcher member which
should be used to dispatch work (in particular, handling incoming packets in
the implementation of remote EPC implementations like SimpleRemoteEPC).
The GenericNamedTask template can be used to wrap function objects that are
callable as 'void()' (along with an optional name to describe the task).
The makeGenericNamedTask functions can be used to create GenericNamedTask
instances without having to name the function object type.
In a future patch ExecutionSession will be updated to use the
ExecutorProcessControl's dispatcher, instead of its DispatchTaskFunction.
|
|
This moves the registry higher in the LLVM library dependency stack.
Every client of the target registry needs to link against MC anyway to
actually use the target, so we might as well move this out of Support.
This allows us to ensure that Support doesn't have includes from MC/*.
Differential Revision: https://reviews.llvm.org/D111454
|
|
The page size for the host process is only needed in the in-process use case.
|
|
ORC errors preserve the SymbolStringPool since 6fe2e9a9cc8, so we can stop
bailing out early.
|
