| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
This PR is a part of the effort to make the VFS used in the compiler
more explicit and consistent.
Instead of creating the VFS deep within the compiler (in
`CompilerInstance::createFileManager()`), clients are now required to
explicitly call `CompilerInstance::createVirtualFileSystem()` and
provide the base VFS from the outside.
This PR also helps in breaking up the dependency cycle where creating a
properly configured `DiagnosticsEngine` requires a properly configured
VFS, but creating properly configuring a VFS requires the
`DiagnosticsEngine`.
Both `CompilerInstance::create{FileManager,Diagnostics}()` now just use
the VFS already in `CompilerInstance` instead of taking one as a
parameter, making the VFS consistent across the instance sub-object.
|
|
(#157358)
Introduce a custom lambda mechanism that allows injecting user-defined
code into the Remote JIT’s executor.
---------
Co-authored-by: kr-2003 <kumar.kr.abhinav@gmail.com>
|
|
This is necessary when using ASan, since the larger code size will lead
to errors such as:
```
JIT session error: In graph clojure_core-clojure.core$clojure_core_cpp_cast_24538-24543-jitted-objectbuffer, section .eh_frame: relocation target 0x7bffe374b000 (DW.ref.__gxx_personality_v0) is out of range of Delta32 fixup at address 0x7bffe374b000 (<anonymous block> @ 0x7fffebf48158 + 0x13)
```
Previously, `clang::Interpreter` would hard-code the usage of a small
code model. With this change, we default to small, but allow for custom
values. This related to #102858 and #135401.
There is no change to default behavior here.
@lhames for review.
|
|
This is a refactoring PR. It sinks RemoteJITUtils into Interpreter and IncrementalExecutor classes.
---------
Co-authored-by: kr-2003 <kumar.kr.abhinav@gmail.com>
|
|
IncrementalParser (#137458)
Read discussion : https://github.com/llvm/llvm-project/pull/136404#discussion_r2059149768
and the following comments for context
Motivation
1) `IncrementalAction` is designed to keep Frontend statealive across
inputs. As per the docstring: “IncrementalAction ensures it keeps its
underlying action's objects alive as long as the IncrementalParser needs
them.”
2) To align responsibilities with that contract, the parser layer (host:
`IncrementalParser`, device: `IncrementalCUDADeviceParser`) should
manage PTU registration and module generation, while the interpreter
orchestrates at a higher level.
What this PR does
1) Moves CodeGen surfaces behind IncrementalAction:
GenModule(), getCodeGen(), and the cached “first CodeGen module” now
live in IncrementalAction.
2) Moves PTU ownership to the parser layer:
Adds IncrementalParser::RegisterPTU(…) (and device counterpart)
3) Add device-side registration in IncrementalCUDADeviceParser.
4) Remove Interpreter::{getCodeGen, GenModule, RegisterPTU}.
|
|
Handles clang::DiagnosticsEngine and clang::DiagnosticIDs.
For DiagnosticIDs, this mostly migrates from `new DiagnosticIDs` to
convenience method `DiagnosticIDs::create()`.
Part of cleanup https://github.com/llvm/llvm-project/issues/151026
|
|
Updated error message logic for undo function. Throws different errors
for the case of there being nothing to undo, and for the case of
requesting more undos than there are operations to undo.
Fixes https://github.com/llvm/llvm-project/issues/143668
|
|
The idea is to store a type-value pair in clang::Value which is updated
by the interpreter runtime. The class copies builtin types and boxes
non-builtin types to provide some lifetime control.
The patch enables default printers for C and C++ using a very
minimalistic approach. We handle enums, arrays and user types. Once we
land this we can focus on enabling user-defined pretty-printers which
take control over printing of types
The work started as part of https://reviews.llvm.org/D146809, then we
created a giant in https://github.com/llvm/llvm-project/pull/84769
|
|
(#146620)
This patch fixes an issue where Microsoft-specific layout attributes,
such as __declspec(empty_bases), were ignored during CUDA/HIP device
compilation on a Windows host. This caused a critical memory layout
mismatch between host and device objects, breaking libraries that rely
on these attributes for ABI compatibility.
The fix introduces a centralized hasMicrosoftRecordLayout() check within
the TargetInfo class. This check is aware of the auxiliary (host) target
and is set during TargetInfo::adjust if the host uses a Microsoft ABI.
The empty_bases, layout_version, and msvc::no_unique_address attributes
now use this centralized flag, ensuring device code respects them and
maintains layout consistency with the host.
Fixes: https://github.com/llvm/llvm-project/issues/146047
|
|
This removes ThreadSafeContext::Lock, ThreadSafeContext::getLock, and
ThreadSafeContext::getContext, and replaces them with a
ThreadSafeContext::withContextDo method (and const override).
The new method can be used to access an existing
ThreadSafeContext-wrapped LLVMContext in a safe way:
ThreadSafeContext TSCtx = ... ;
TSCtx.withContextDo([](LLVMContext *Ctx) {
// this closure has exclusive access to Ctx.
});
The new API enforces correct locking, whereas the old APIs relied on
manual locking (which almost no in-tree code preformed, relying instead
on incidental exclusive access to the ThreadSafeContext).
|
|
(#144064)
This PR introduces out-of-process (OOP) execution support for
Clang-Repl. With this enhancement, two new flags, oop-executor and
oop-executor-connect, are added to the Clang-Repl interface. These flags
enable the launch of an external executor (llvm-jitlink-executor), which
handles code execution in a separate process.
|
|
(#139584)"
This reverts commit e2a885537f11f8d9ced1c80c2c90069ab5adeb1d. Build failures were fixed right away and reverting the original commit without the fixes breaks the build again.
|
|
(#139584)"
This reverts commit 9e306ad4600c4d3392c194a8be88919ee758425c.
Multiple builtbot failures have been reported:
https://github.com/llvm/llvm-project/pull/139584
|
|
The `DiagnosticOptions` class is currently intrusively
reference-counted, which makes reasoning about its lifetime very
difficult in some cases. For example, `CompilerInvocation` owns the
`DiagnosticOptions` instance (wrapped in `llvm::IntrusiveRefCntPtr`) and
only exposes an accessor returning `DiagnosticOptions &`. One would
think this gives `CompilerInvocation` exclusive ownership of the object,
but that's not the case:
```c++
void shareOwnership(CompilerInvocation &CI) {
llvm::IntrusiveRefCntPtr<DiagnosticOptions> CoOwner = &CI.getDiagnosticOptions();
// ...
}
```
This is a perfectly valid pattern that is being actually used in the
codebase.
I would like to ensure the ownership of `DiagnosticOptions` by
`CompilerInvocation` is guaranteed to be exclusive. This can be
leveraged for a copy-on-write optimization later on. This PR changes
usages of `DiagnosticOptions` across `clang`, `clang-tools-extra` and
`lldb` to not be intrusively reference-counted.
|
|
|
|
Check this error for more context
(https://github.com/compiler-research/CppInterOp/actions/runs/14749797085/job/41407625681?pr=491#step:10:531)
This fails with
```
* thread #1, name = 'CppInterOpTests', stop reason = signal SIGSEGV: address not mapped to object (fault address: 0x55500356d6d3)
* frame #0: 0x00007fffee41cfe3 libclangCppInterOp.so.21.0gitclang::PragmaNamespace::~PragmaNamespace() + 99
frame #1: 0x00007fffee435666 libclangCppInterOp.so.21.0gitclang::Preprocessor::~Preprocessor() + 3830
frame #2: 0x00007fffee20917a libclangCppInterOp.so.21.0gitstd::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() + 58
frame #3: 0x00007fffee224796 libclangCppInterOp.so.21.0gitclang::CompilerInstance::~CompilerInstance() + 838
frame #4: 0x00007fffee22494d libclangCppInterOp.so.21.0gitclang::CompilerInstance::~CompilerInstance() + 13
frame #5: 0x00007fffed95ec62 libclangCppInterOp.so.21.0gitclang::IncrementalCUDADeviceParser::~IncrementalCUDADeviceParser() + 98
frame #6: 0x00007fffed9551b6 libclangCppInterOp.so.21.0gitclang::Interpreter::~Interpreter() + 102
frame #7: 0x00007fffed95598d libclangCppInterOp.so.21.0gitclang::Interpreter::~Interpreter() + 13
frame #8: 0x00007fffed9181e7 libclangCppInterOp.so.21.0gitcompat::createClangInterpreter(std::vector<char const*, std::allocator<char const*>>&) + 2919
```
Problem :
1) The destructor currently handles no clearance for the DeviceParser
and the DeviceAct. We currently only have this
https://github.com/llvm/llvm-project/blob/976493822443c52a71ed3c67aaca9a555b20c55d/clang/lib/Interpreter/Interpreter.cpp#L416-L419
2) The ownership for DeviceCI currently is present in
IncrementalCudaDeviceParser. But this should be similar to how the
combination for hostCI, hostAction and hostParser are managed by the
Interpreter. As on master the DeviceAct and DeviceParser are managed by
the Interpreter but not DeviceCI. This is problematic because :
IncrementalParser holds a Sema& which points into the DeviceCI. On
master, DeviceCI is destroyed before the base class ~IncrementalParser()
runs, causing Parser::reset() to access a dangling Sema (and as Sema
holds a reference to Preprocessor which owns PragmaNamespace) we see
this
```
* frame #0: 0x00007fffee41cfe3 libclangCppInterOp.so.21.0gitclang::PragmaNamespace::~PragmaNamespace() + 99
frame #1: 0x00007fffee435666 libclangCppInterOp.so.21.0gitclang::Preprocessor::~Preprocessor() + 3830
```
|
|
This PR hides the reference-counted pointer that holds `TargetOptions`
from the public API of `CompilerInvocation`. This gives
`CompilerInvocation` an exclusive control over the lifetime of this
member, which will eventually be leveraged to implement a copy-on-write
behavior.
There are two clients that currently share ownership of that pointer:
* `TargetInfo` - This was refactored to hold a non-owning reference to
`TargetOptions`. The options object is typically owned by the
`CompilerInvocation` or by the new `CompilerInstance::AuxTargetOpts` for
the auxiliary target. This needed a bit of care in `ASTUnit::Parse()` to
keep the `CompilerInvocation` alive.
* `clangd::PreambleData` - This was refactored to exclusively own the
`TargetOptions` that get moved out of the `CompilerInvocation`.
|
|
`clang-repl --cuda` was previously crashing with a segmentation fault,
instead of reporting a clean error
```
(base) anutosh491@Anutoshs-MacBook-Air bin % ./clang-repl --cuda
#0 0x0000000111da4fbc llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/opt/local/libexec/llvm-20/lib/libLLVM.dylib+0x150fbc)
#1 0x0000000111da31dc llvm::sys::RunSignalHandlers() (/opt/local/libexec/llvm-20/lib/libLLVM.dylib+0x14f1dc)
#2 0x0000000111da5628 SignalHandler(int) (/opt/local/libexec/llvm-20/lib/libLLVM.dylib+0x151628)
#3 0x000000019b242de4 (/usr/lib/system/libsystem_platform.dylib+0x180482de4)
#4 0x0000000107f638d0 clang::IncrementalCUDADeviceParser::IncrementalCUDADeviceParser(std::__1::unique_ptr<clang::CompilerInstance, std::__1::default_delete<clang::CompilerInstance>>, clang::CompilerInstance&, llvm::IntrusiveRefCntPtr<llvm::vfs::InMemoryFileSystem>, llvm::Error&, std::__1::list<clang::PartialTranslationUnit, std::__1::allocator<clang::PartialTranslationUnit>> const&) (/opt/local/libexec/llvm-20/lib/libclang-cpp.dylib+0x216b8d0)
#5 0x0000000107f638d0 clang::IncrementalCUDADeviceParser::IncrementalCUDADeviceParser(std::__1::unique_ptr<clang::CompilerInstance, std::__1::default_delete<clang::CompilerInstance>>, clang::CompilerInstance&, llvm::IntrusiveRefCntPtr<llvm::vfs::InMemoryFileSystem>, llvm::Error&, std::__1::list<clang::PartialTranslationUnit, std::__1::allocator<clang::PartialTranslationUnit>> const&) (/opt/local/libexec/llvm-20/lib/libclang-cpp.dylib+0x216b8d0)
#6 0x0000000107f6bac8 clang::Interpreter::createWithCUDA(std::__1::unique_ptr<clang::CompilerInstance, std::__1::default_delete<clang::CompilerInstance>>, std::__1::unique_ptr<clang::CompilerInstance, std::__1::default_delete<clang::CompilerInstance>>) (/opt/local/libexec/llvm-20/lib/libclang-cpp.dylib+0x2173ac8)
#7 0x000000010206f8a8 main (/opt/local/libexec/llvm-20/bin/clang-repl+0x1000038a8)
#8 0x000000019ae8c274
Segmentation fault: 11
```
The underlying issue was that the `DeviceCompilerInstance` (used for
device-side CUDA compilation) was never initialized with a `Sema`, which
is required before constructing the `IncrementalCUDADeviceParser`.
https://github.com/llvm/llvm-project/blob/89687e6f383b742a3c6542dc673a84d9f82d02de/clang/lib/Interpreter/DeviceOffload.cpp#L32
https://github.com/llvm/llvm-project/blob/89687e6f383b742a3c6542dc673a84d9f82d02de/clang/lib/Interpreter/IncrementalParser.cpp#L31
Unlike the host-side `CompilerInstance` which runs `ExecuteAction`
inside the Interpreter constructor (thereby setting up Sema), the
device-side CI was passed into the parser uninitialized, leading to an
assertion or crash when accessing its internals.
To fix this, I refactored the `Interpreter::create` method to include an
optional `DeviceCI` parameter. If provided, we know we need to take care
of this instance too. Only then do we construct the
`IncrementalCUDADeviceParser`.
|
|
|
|
(#133037)
**Currently we don't make use of the JIT for the wasm use cases so the
approach using the execution engine won't work in these cases.**
Rather if we use dlopen. We should be able to do the following
(demonstrating through a toy project)
1) Make use of LoadDynamicLibrary through the given implementation
```
extern "C" EMSCRIPTEN_KEEPALIVE int load_library(const char *name) {
auto Err = Interp->LoadDynamicLibrary(name);
if (Err) {
llvm::logAllUnhandledErrors(std::move(Err), llvm::errs(), "load_library error: ");
return -1;
}
return 0;
}
```
2) Add a button to call load_library once the library has been added in
our MEMFS (currently we have symengine built as a SIDE MODULE and we are
loading it)
|
|
libraries. (#117475)"
This reverts commit 30ad53b92cec0cff9679d559edcc5b933312ba0c as it breaks
systems that don't have a systemwide libc++ or libstdc++ installed. It should
be rewritten to not invoke the system linker. In the meanwhile, reverting
to unblock the bots.
|
|
(#117475)
Apply the fix suggested by Lang Hames to address a crash in Clang-REPL
that occurs during the execution of `__run_exit_handlers` when using
dynamic libraries.
|
|
browser (#118107)
While running clang-repl in the browser, we would be interested in this
cc1 command
`
"" -cc1 -triple wasm32-unknown-emscripten -emit-obj -disable-free
-clear-ast-before-backend -disable-llvm-verifier -discard-value-names
-main-file-name "<<< inputs >>>" -mrelocation-model static
-mframe-pointer=none -ffp-contract=on -fno-rounding-math
-mconstructor-aliases -target-cpu generic -debugger-tuning=gdb
-fdebug-compilation-dir=/ -v -fcoverage-compilation-dir=/ -resource-dir
/lib/clang/19 -internal-isystem /include/wasm32-emscripten/c++/v1
-internal-isystem /include/c++/v1 -internal-isystem
/lib/clang/19/include -internal-isystem /include/wasm32-emscripten
-internal-isystem /include -std=c++17 -fdeprecated-macro -ferror-limit
19 -fvisibility=default -fgnuc-version=4.2.1 -fskip-odr-check-in-gmf
-fcxx-exceptions -fexceptions -fincremental-extensions -o "<<< inputs
>>>.o" -x c++ "<<< inputs >>>"
`
As can be seen `shared` is anyway overwritten by `static` which is also
what would be provided by default. Hence we can get rid of the shared
flag here.
|
|
browser (#117978)
Co-authored-by: Vassil Vassilev <v.g.vassilev@gmail.com>
|
|
This reverts commit a1153cd6fedd4c906a9840987934ca4712e34cb2 with fixes
to lldb breakages.
Fixes https://github.com/llvm/llvm-project/issues/117145.
|
|
Reverted for causing:
https://github.com/llvm/llvm-project/issues/117145
This reverts commit bdd10d9d249bd1c2a45e3de56a5accd97e953458.
|
|
Starting with 41e3919ded78d8870f7c95e9181c7f7e29aa3cc4 DiagnosticsEngine
creation might perform IO. It was implicitly defaulting to
getRealFileSystem. This patch makes it explicit by pushing the decision
making to callers.
It uses ambient VFS if one is available, and keeps using
`getRealFileSystem` if there aren't any VFS.
|
|
This patch improves the code reuse of the actions system and adds
several improvements for easier debugging via clang-repl
--debug-only=clang-repl.
The change inimproves the consistency of the TUKind when actions are
handled within a WrapperFrontendAction. In this case instead of falling
back to default TU_Complete, we forward to the TUKind of the ASTContext
which presumably was created by the intended action. This enables the
incremental infrastructure to reuse code.
This patch also clones the first llvm::Module because the first PTU now
can come from -include A.h and the presumption of llvm::Module being
empty does not hold. The changes are a first step to fix the issues with
`clang-repl --cuda`.
|
|
(#116735)
There are a couple changes in this PR that help getting clang-repl to
run in the browser. Using a jupyterlite instance for the example pasted
below
1) Updating flags responsible for generating shared wasm binaries that
need to be dynamically loaded Most Importantly as can be seen in the
changes `shared` and `allow-undefined` are crucial.
2) While exiting we encounter this.
Now as can be seen here
https://github.com/llvm/llvm-project/blob/cd418030de7ae75750bc4e48d1238baf03c675e5/clang/lib/Interpreter/Interpreter.cpp#L421-L430
We call cleanUP in the destructor. Now cleanUP through
IncrementalExecutor tries to deinitialize the JIT which wasn't even
intialized as runCtors in wasm.cpp is a no-op
https://github.com/llvm/llvm-project/blob/cd418030de7ae75750bc4e48d1238baf03c675e5/clang/lib/Interpreter/IncrementalExecutor.cpp#L94-L101
https://github.com/llvm/llvm-project/blob/cd418030de7ae75750bc4e48d1238baf03c675e5/clang/lib/Interpreter/Wasm.cpp#L107-L109
|
|
Reverts llvm/llvm-project#110418
Buildbot encountered a failure.
|
|
This PR introduces out-of-process (OOP) execution support for
Clang-Repl. With this enhancement, two new flags, `oop-executor` and
`oop-executor-connect`, are added to the Clang-Repl interface. These
flags enable the launch of an external executor
(`llvm-jitlink-executor`), which handles code execution in a separate
process.
|
|
(#107737)
This patch improves the design of the IncrementalParser and Interpreter
classes. Now the incremental parser is only responsible for building the
partial translation unit declaration and the AST, while the Interpreter
fills in the lower level llvm::Module and other JIT-related
infrastructure. Finally the Interpreter class now orchestrates the AST
and the LLVM IR with the IncrementalParser and IncrementalExecutor
classes.
The design improvement allows us to rework some of the logic that
extracts an interpreter value into the clang::Value object. The new
implementation simplifies use-cases which are used for out-of-process
execution by allowing interpreter to be inherited or customized with an
clang::ASTConsumer.
This change will enable completing the pretty printing work which is in
llvm/llvm-project#84769
|
|
(#99599)
Close https://github.com/llvm/llvm-project/issues/99479
See https://github.com/llvm/llvm-project/issues/99479 for details
|
|
When generating runtime interface bindings, extend integral types to the
native register size rather than 64-bit per se
Fixes #94994
|
|
This commit introduces support for running clang-repl and executing C++
code interactively inside a Javascript engine using WebAssembly when
built with Emscripten. This is achieved by producing WASM "shared
libraries" that can be loaded by the Emscripten runtime using dlopen()
More discussion is available in https://reviews.llvm.org/D158140
Co-authored-by: Anubhab Ghosh <anubhabghosh.me@gmail.com>
|
|
|
|
Until now the IncrExecutor was created lazily on the first execution
request. In order to process the PTUs that come from initialization, we
have to do it upfront implicitly.
|
|
This patch converts the enum into scoped enum, and moves it into its own header for the time being. It's definition is needed in `Sema.h`, and is going to be needed in upcoming `SemaObjC.h`. `Lookup.h` can't hold it, because it includes `Sema.h`.
|
|
derived classes (#84461)
The LLJITBuilder interface provides a very convenient way to configure
the ORCv2 JIT engine. IncrementalExecutor already used it internally to
construct the JIT, but didn't provide external access. This patch lifts
control of the creation process to the Interpreter and allows injection
of a custom instance through the extended interface. The Interpreter's
default behavior remains unchanged and the IncrementalExecutor remains
an implementation detail.
|
|
Interpreter interface (#84460)
IncrementalExecutor is an implementation detail of the Interpreter. In
order to test extended features properly, we must be able to setup and
tear down the executor manually.
|
|
The Interpreter locks PTUs that originate from implicit runtime code and
initialization to prevent users from undoing them accidentally.
The previous implementation seemed hacky, because it required the reader
to be familiar with the internal workings of the PTU stack. The concept
itself is a pragmatic solution and not very surprising. This patch
introduces a function for it and adds a comment.
|
|
RuntimeInterfaceBuilder wires up JITed expressions with the hardcoded
Interpreter runtime. It's used only for value printing right now, but it
is not limited to that. The default implementation focuses on an
evaluation process where the Interpreter has direct access to the memory
of JITed expressions (in-process execution or shared memory).
We need a different approach to support out-of-process evaluation or
variations of the runtime. It seems reasonable to expose a minimal
interface for it. The new RuntimeInterfaceBuilder is an abstract base
class in the public header. For that, the TypeVisitor had to become a
component (instead of inheriting from it). FindRuntimeInterface() was
adjusted to return an instance of the RuntimeInterfaceBuilder and it can
be overridden from derived classes.
|
|
(#84174)
With out-of-process execution the target triple can be different from
the one on the host. We need an interface to configure it.
Relanding this with cleanup-fixes in the unittest.
|
|
argument" (#84261)
Reverts llvm/llvm-project#84174 due too sanitizer memory leak detection
|
|
With out-of-process execution the target triple can be different from
the one on the host. We need an interface to configure it.
|
|
Reverting because `clang-repl -Xcc -mcpu=arm1176jzf-s` isn't overwriting
this as I had expected. We need to check whether a specific CPU flag was
given by the user first.
Reverts llvm/llvm-project#77491
|
|
(#76218)
This patch brings back the basic support for C by inserting the required
for value printing runtime only when we are in C++ mode. Additionally,
it defines a new overload of operator placement new because we can't
really forward declare it in a library-agnostic way.
Fixes the issue described in llvm/llvm-project#69072.
|
|
We can pass `-mcpu=native` to the clang driver to let it consider the
host CPU when choosing the compile target for `clang-repl`. We can
already achieve this behavior with `clang-repl -Xcc -mcpu=native`, but
it seems like a reasonable default actually.
The trade-off between optimizing for a specific CPU and maximum
compatibility often leans towards the latter for static binaries,
because distributing many versions is cumbersome. However, when
compiling at runtime, we know the exact target CPU and we can use that
to optimize the generated code.
This patch makes a difference especially for "scattered" architectures
like ARM. When cross-compiling for a Raspberry Pi for example, we may
use a stock toolchain like arm-linux-gnueabihf-gcc. The resulting binary
will be compatible with all hardware versions. This is handy, but they
will all have `arm-linux-gnueabihf` as their host triple. Previously,
this caused the clang driver to select triple `armv6kz-linux-gnueabihf`
and CPU `arm1176jzf-s` as the REPL target. After this patch the default
triple and CPU on Raspberry Pi 4b will be `armv8a-linux-gnueabihf` and
`cortex-a72` respectively.
With this patch clang-repl matches the host detection in Orc.
|
|
This patch contains changes from
002d471a4a3cd8b429e4ca7c84fd54a642e50e4c, in
addition to a bug fix that added a virtual destructor to
`CompletionContextHandler`
The original changes in the orginal commit piggybacks on clang's
semantic modules to enable semantic completion. In particular, we use
`CodeCompletionContext` to differentiate two types of code completion.
We also
extract the relevant type information from it.
|
|
Reverts llvm/llvm-project#67349
There are some issues with the sanitizers. We will reland once that's fixed.
|
