| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
This PR virtualizes module cache pruning via the new `ModuleCache`
interface. Currently this is an NFC, but I left a FIXME in
`InProcessModuleCache` to make this more efficient for the dependency
scanner.
|
|
This PR changes `llvm::FileCollector` to use the `llvm::vfs::FileSystem`
API for making file paths absolute instead of using
`llvm::sys::fs::make_absolute()` directly. This matches the behavior of
the compiler on most other input files.
|
|
This PR is a part of the effort to make the VFS used in the compiler
more explicit and consistent.
Instead of creating the VFS deep within the compiler (in
`CompilerInstance::createFileManager()`), clients are now required to
explicitly call `CompilerInstance::createVirtualFileSystem()` and
provide the base VFS from the outside.
This PR also helps in breaking up the dependency cycle where creating a
properly configured `DiagnosticsEngine` requires a properly configured
VFS, but creating properly configuring a VFS requires the
`DiagnosticsEngine`.
Both `CompilerInstance::create{FileManager,Diagnostics}()` now just use
the VFS already in `CompilerInstance` instead of taking one as a
parameter, making the VFS consistent across the instance sub-object.
|
|
This PR uses the new-ish `llvm::vfs::FileSystem::visit()` interface to
collect VFS overlay entries from an existing `FileSystem` instance
rather than parsing the VFS YAML file anew. This prevents duplicate
diagnostics as observed by `clang/test/VFS/broken-vfs-module-dep.c`.
|
|
This reverts commit 613caa909c78f707e88960723c6a98364656a926, essentially
reapplying 4a4bddec3571d78c8073fa45b57bbabc8796d13d after moving
`normalizeModuleCachePath` from clangFrontend to clangLex.
This PR is part of an effort to remove file system usage from the
command line parsing code. The reason for that is that it's impossible
to do file system access correctly without a configured VFS, and the VFS
can only be configured after the command line is parsed. I don't want to
intertwine command line parsing and VFS configuration, so I decided to
perform the file system access after the command line is parsed and the
VFS is configured - ideally right before the file system entity is used
for the first time.
This patch delays normalization of the module cache path until
`CompilerInstance` is asked for the cache path in the current
compilation context.
|
|
This reverts commit 4a4bddec3571d78c8073fa45b57bbabc8796d13d. The Serialization library doesn't link Frontend, where CompilerInstance lives, causing link failures on some build bots.
|
|
This PR is part of an effort to remove file system usage from the
command line parsing code. The reason for that is that it's impossible
to do file system access correctly without a configured VFS, and the VFS
can only be configured after the command line is parsed. I don't want to
intertwine command line parsing and VFS configuration, so I decided to
perform the file system access after the command line is parsed and the
VFS is configured - ideally right before the file system entity is used
for the first time.
This patch delays normalization of the module cache path until
`CompilerInstance` is asked for the cache path in the current
compilation context.
|
|
Adopt new virtual output backend in CompilerInstance.
|
|
Upstreams https://github.com/swiftlang/llvm-project/pull/10943.
https://github.com/llvm/llvm-project/pull/134887 added a clone for the
compiler instance in `compileModuleAndReadASTImpl`, which would then be
destroyed *after* the corresponding read in the importing instance.
Swift has a `SwiftNameLookupExtension` module extension which updates
(effectively) global state - populating the lookup table for a module on
read and removing it when the module is destroyed.
With newly cloned instance, we would then see:
- Module compiled with cloned instance
- Module read with importing instance
- Lookup table for that module added
- Cloned instance destroyed
- Module from that cloned instance destroyed
- Lookup table for that module name removed
Depending on the original semantics is incredibly fragile, but for now
it's good enough to ensure that the read in the importing instance is
after the cloned instanced is destroyed. Ideally we'd only ever add to
the lookup tables in the original importing instance, never its clones.
Co-authored-by: Ben Barham <ben_barham@apple.com>
|
|
Finishes (#151774)
When two threads are accessing the same `pcm`, it is possible that the
reading thread sees the timestamp update, while the file on disk is not
updated.
This PR moves timestamp update from `writeAST` to
`compileModuleAndReadASTImpl`, so we only update the timestamp after the
file has been committed to disk.
rdar://152097193
|
|
This commit handles the following types:
- clang::ExternalASTSource
- clang::TargetInfo
- clang::ASTContext
- clang::SourceManager
- clang::FileManager
Part of cleanup #151026
|
|
Handles clang::DiagnosticsEngine and clang::DiagnosticIDs.
For DiagnosticIDs, this mostly migrates from `new DiagnosticIDs` to
convenience method `DiagnosticIDs::create()`.
Part of cleanup https://github.com/llvm/llvm-project/issues/151026
|
|
llvm::vfs::FileSystem. (#151407)
This switches to `makeIntrusiveRefCnt<FileSystem>` where creating a new
object, and to passing/returning by `IntrusiveRefCntPtr<FileSystem>`
instead of `FileSystem*` or `FileSystem&`, when dealing with existing
objects.
Part of cleanup #151026.
|
|
Reverts llvm/llvm-project#147959
|
|
report_fatal_error is not a good way to report diagnostics to the users, so this switches to using actual diagnostic reporting mechanisms instead.
Fixes #147187
|
|
Some `LangOptions` duplicate their `CodeGenOptions` counterparts. My
understanding is that this was done solely because some infrastructure
(like preprocessor initialization, serialization, module compatibility
checks, etc.) were only possible/convenient for `LangOptions`. This PR
implements the missing support for `CodeGenOptions`, which makes it
possible to remove some duplicate `LangOptions` fields and simplify the
logic. Motivated by https://github.com/llvm/llvm-project/pull/146342.
|
|
(#146620)
This patch fixes an issue where Microsoft-specific layout attributes,
such as __declspec(empty_bases), were ignored during CUDA/HIP device
compilation on a Windows host. This caused a critical memory layout
mismatch between host and device objects, breaking libraries that rely
on these attributes for ABI compatibility.
The fix introduces a centralized hasMicrosoftRecordLayout() check within
the TargetInfo class. This check is aware of the auxiliary (host) target
and is set during TargetInfo::adjust if the host uses a Microsoft ABI.
The empty_bases, layout_version, and msvc::no_unique_address attributes
now use this centralized flag, ensuring device code respects them and
maintains layout consistency with the host.
Fixes: https://github.com/llvm/llvm-project/issues/146047
|
|
Static analysis flagged Input as a large object that would benefit from
being moved over being copied.
|
|
These are identified by misc-include-cleaner. I've filtered out those
that break builds. Also, I'm staying away from llvm-config.h,
config.h, and Compiler.h, which likely cause platform- or
compiler-specific build failures.
|
|
(#139584)"
This reverts commit e2a885537f11f8d9ced1c80c2c90069ab5adeb1d. Build failures were fixed right away and reverting the original commit without the fixes breaks the build again.
|
|
(#139584)"
This reverts commit 9e306ad4600c4d3392c194a8be88919ee758425c.
Multiple builtbot failures have been reported:
https://github.com/llvm/llvm-project/pull/139584
|
|
The `DiagnosticOptions` class is currently intrusively
reference-counted, which makes reasoning about its lifetime very
difficult in some cases. For example, `CompilerInvocation` owns the
`DiagnosticOptions` instance (wrapped in `llvm::IntrusiveRefCntPtr`) and
only exposes an accessor returning `DiagnosticOptions &`. One would
think this gives `CompilerInvocation` exclusive ownership of the object,
but that's not the case:
```c++
void shareOwnership(CompilerInvocation &CI) {
llvm::IntrusiveRefCntPtr<DiagnosticOptions> CoOwner = &CI.getDiagnosticOptions();
// ...
}
```
This is a perfectly valid pattern that is being actually used in the
codebase.
I would like to ensure the ownership of `DiagnosticOptions` by
`CompilerInvocation` is guaranteed to be exclusive. This can be
leveraged for a copy-on-write optimization later on. This PR changes
usages of `DiagnosticOptions` across `clang`, `clang-tools-extra` and
`lldb` to not be intrusively reference-counted.
|
|
This reverts commit 7a242387c950c7060143da6da0e6fb91f36bb458. Even after 175f8a44, the Modules/fmodules-validate-once-per-build-session.c test is not fixed on the clang-armv8-quick build bot. (Failure occurs on line 114.)
|
|
This reverts commit 18b885f66babff3a10451bc811ffc077d61ed8ee, effectively reapplying #139987. This commit fixes unit tests (for example ASTUnitTest.SaveLoadPreservesLangOptionsInPrintingPolicy) where the `ASTUnit::ModCache` pointer dereferenced within `ASTUnit::serialize()` was null. This commit makes sure each factory function does initialize `ASTUnit::ModCache`.
|
|
Reverts llvm/llvm-project#138983
|
|
Timestamps are an implementation detail of the cross-process module
cache implementation. This PR hides it from the `ModuleCache` API, which
simplifies the in-process implementation.
|
|
Force-validation of user headers was implemented in acb803e8 to deal
with files changing during build. The dependency scanner guarantees an
immutable file system during single build session, so the validation is
unnecessary. (We don't hit the disk too often due to the caching VFS,
but even avoiding going to the cache and deserializing the input files
makes sense.)
|
|
Instead of eagerly populating the `clang::ModuleMap` when looking up a
module by name, this patch changes `HeaderSearch` to only load the
modules that are actually used.
This introduces `ModuleMap::findOrLoadModule` which will load modules
from parsed but not loaded module maps. This cannot be used anywhere
that the module loading code calls into as it can create infinite
recursion.
This currently just reparses module maps when looking up a module by
header. This is fine as redeclarations are allowed from the same file,
but future patches will also make looking up a module by header lazy.
This patch changes the shadow.m test to use explicitly built modules and
`#import`. This test and the shadow feature are very brittle and do not
work in general. The test relied on pcm files being left behind by prior
failing clang invocations that were then reused by the last invocation.
If you clean the cache then the last invocation will always fail. This
is because the input module map and the `-fmodule-map-file=` module map
are parsed in the same module scope, and `-fmodule-map-file=` is
forwarded to implicit module builds. That means you are guaranteed to
hit a module redeclaration error if the TU actually imports the module
it is trying to shadow.
This patch changes when we load A2's module map to after the `A` module
has been loaded, which sets the `IsFromModuleFile` bit on `A`. This
means that A2's `A` is skipped entirely instead of creating a shadow
module, and we get textual inclusion. It is possible to construct a case
where this would happen before this patch too.
An upcoming patch in this series will rework shadowing to work in the
general case, but that's only possible once header -> module lookup is
lazy too.
|
|
This PR makes it so that `CompilerInvocation` needs to be provided to
`CompilerInstance` on construction. There are a couple of benefits in my
view:
* Making it impossible to mis-use some `CompilerInstance` APIs. For
example there are cases, where `createDiagnostics()` was called before
`setInvocation()`, causing the `DiagnosticEngine` to use the
default-constructed `DiagnosticOptions` instead of the intended ones.
* This shrinks `CompilerInstance`'s state space.
* This makes it possible to access **the** invocation in
`CompilerInstance`'s constructor (to be used in a follow-up).
|
|
to get more stack space (#136046)
Reland https://github.com/llvm/llvm-project/pull/133173
Clang spawns a new thread to avoid running out of stack space. This can
make debugging and performance analysis more difficult as how the
threads are connected is difficult to recover.
This patch introduces `runOnNewStack` and applies it in Clang. On
platforms that have good support for it this allocates a new stack and
moves to it using assembly. Doing split stacks like this actually runs
on most platforms, but many debuggers and unwinders reject the large or
backwards stack offsets that occur. Apple platforms and tools are known
to support this, so this only enables it there for now.
|
|
thread-safe (#137227)
The same principle as #135473, #135737, #136178, #136601 & #137059.
|
|
This PR hides the reference-counted pointer that holds `TargetOptions`
from the public API of `CompilerInvocation`. This gives
`CompilerInvocation` an exclusive control over the lifetime of this
member, which will eventually be leveraged to implement a copy-on-write
behavior.
There are two clients that currently share ownership of that pointer:
* `TargetInfo` - This was refactored to hold a non-owning reference to
`TargetOptions`. The options object is typically owned by the
`CompilerInvocation` or by the new `CompilerInstance::AuxTargetOpts` for
the auxiliary target. This needed a bit of care in `ASTUnit::Parse()` to
keep the `CompilerInvocation` alive.
* `clangd::PreambleData` - This was refactored to exclusively own the
`TargetOptions` that get moved out of the `CompilerInvocation`.
|
|
This PR makes another piece of the
`CompilerInstance::cloneForModuleCompile()` result thread-safe: the
module build stack. This data structure is used to detect cyclic
dependencies between modules. The problem is that it uses
`FullSourceLoc` which refers to the `SourceManager` of the parent
`CompilerInstance`: if two threads happen to execute `CompilerInstance`s
cloned from the same parent concurrently, and both discover a dependency
cycle, they may concurrently access the parent `SourceManager` when
emitting the diagnostic, creating a data race.
In this PR, we prevent this by keeping the stack empty and moving the
responsibility of cycle detection to the client. The client can recreate
the same module build stack externally and ensure thread-safety by
enforcing mutual exclusion.
|
|
This PR fixes two issues in one go:
1. The dependency directives getter (a `std::function`) was being stored
in `PreprocessorOptions`. This goes against the principle where the
options classes are supposed to be value-objects representing the `-cc1`
command line arguments. This is fixed by moving the getter directly to
`CompilerInstance` and propagating it explicitly.
2. The getter was capturing the `ScanInstance` VFS. That's fine in
synchronous implicit module builds where the same VFS instance is used
throughout, but breaks down once you try to build modules asynchronously
(which forces the use of separate VFS instances). This is fixed by
explicitly passing a `FileManager` into the getter and extracting the
right instance of the scanning VFS out of it.
|
|
The `DiagnosticConsumer` interface is not thread-safe. To enable
thread-safety of `CompilerInstance` objects cloned from the same parent,
this PR allows passing an explicit `DiagnosticConsumer` to
`cloneForModuleCompile()`. This will be used from the dependency
scanner.
|
|
The `llvm::vfs::FileSystem` interface makes no promises around
thread-safety. To enable making `CompilerInstance` thread-safe, this PR
allows passing an explicit VFS to `cloneForModuleCompile()`. This will
be used from the dependency scanner.
|
|
|
|
structures to `IdentifierLoc` (#136077)
This PR reland https://github.com/llvm/llvm-project/pull/135808, fixed
some missed changes in LLDB.
I found this issue when I working on
https://github.com/llvm/llvm-project/pull/107168.
Currently we have many similiar data structures like:
- std::pair<IdentifierInfo *, SourceLocation>.
- Element type of ModuleIdPath.
- IdentifierLocPair.
- IdentifierLoc.
This PR unify these data structures to IdentifierLoc, moved
IdentifierLoc definition to SourceLocation.h, and deleted other similer
data structures.
---------
Signed-off-by: yronglin <yronglin777@gmail.com>
|
|
structures to `IdentifierLoc`" (#135974)
Reverts llvm/llvm-project#135808
Example from the LLDB macOS CI:
https://green.lab.llvm.org/job/llvm.org/view/LLDB/job/as-lldb-cmake/24084/execution/node/54/log/?consoleFull
```
/Users/ec2-user/jenkins/workspace/llvm.org/as-lldb-cmake/llvm-project/lldb/source/Plugins/ExpressionParser/Clang/ClangModulesDeclVendor.cpp:360:49: error: no viable conversion from 'std::pair<clang::IdentifierInfo *, clang::SourceLocation>' to 'clang::ModuleIdPath' (aka 'ArrayRef<IdentifierLoc>')
clang::Module *top_level_module = DoGetModule(clang_path.front(), false);
^~~~~~~~~~~~~~~~~~
/Users/ec2-user/jenkins/workspace/llvm.org/as-lldb-cmake/llvm-project/llvm/include/llvm/ADT/ArrayRef.h:41:40: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'std::pair<clang::IdentifierInfo *, clang::SourceLocation>' to 'const llvm::ArrayRef<clang::IdentifierLoc> &' for 1st argument
class LLVM_GSL_POINTER [[nodiscard]] ArrayRef {
^
/Users/ec2-user/jenkins/workspace/llvm.org/as-lldb-cmake/llvm-project/llvm/include/llvm/ADT/ArrayRef.h:41:40: note: candidate constructor (the implicit move constructor) not viable: no known conversion from 'std::pair<clang::IdentifierInfo *, clang::SourceLocation>' to 'llvm::ArrayRef<clang::IdentifierLoc> &&' for 1st argument
/Users/ec2-user/jenkins/workspace/llvm.org/as-lldb-cmake/llvm-project/llvm/include/llvm/ADT/ArrayRef.h:70:18: note: candidate constructor not viable: no known conversion from 'std::pair<clang::IdentifierInfo *, clang::SourceLocation>' to 'std::nullopt_t' for 1st argument
/*implicit*/ ArrayRef(std::nullopt_t) {}
```
|
|
structures to `IdentifierLoc` (#135808)
I found this issue when I working on
https://github.com/llvm/llvm-project/pull/107168.
Currently we have many similiar data structures like:
- `std::pair<IdentifierInfo *, SourceLocation>`.
- Element type of `ModuleIdPath`.
- `IdentifierLocPair`.
- `IdentifierLoc`.
This PR unify these data structures to `IdentifierLoc`, moved
`IdentifierLoc` definition to SourceLocation.h, and deleted other
similer data structures.
---------
Signed-off-by: yronglin <yronglin777@gmail.com>
|
|
(#135865)
…thread to get more stack space (#133173)"
This change breaks the Clang build on Mac AArch64.
This reverts commit d0c973a7a0149db3b71767d4c5a20a31e6a8ed5b. This
reverts commit 429a84f8a4bf559f43f50072747ef49d3e3b2cf1. This reverts
commit 4f64c80d5a23c244f942193e58ecac666c173308.
|
|
more stack space (#133173)
Clang spawns a new thread to avoid running out of stack space. This can
make debugging and performance analysis more difficult as how the
threads are connected is difficult to recover.
This patch introduces `runOnNewStack` and applies it in Clang. On
platforms that have good support for it this allocates a new stack and
moves to it using assembly. Doing split stacks like this actually runs
on most platforms, but many debuggers and unwinders reject the large or
backwards stack offsets that occur. Apple platforms and tools are known
to support this, so this only enables it there for now.
|
|
This PR makes some progress towards making it possible to create clones
of `CompilerInstance` that are independent of each other and can be used
in a multi-threaded setting. This PR tackles
`CompilerInstance::FailedModules`, makes it a value-type instead of a
mutable shared pointer, and adds explicit copies & moves where
appropriate.
Besides that change, this PR also turns two previously free functions
with internal linkage into member functions of `CompilerInstance`, which
makes it possible to reduce the public API of that class that relates to
`FailedModules`. This reduces some API churn that was necessary for each
new member of `CompilerInstance` that needs to be cloned.
|
|
This PR exposes `cloneForModuleCompile()` as a public `CompilerInstance`
member function. This will be eventually used in the dependency scanner
to customize implicit module builds.
|
|
(#134887)
This PR extracts the creation of `CompilerInstance` for compiling an
implicitly-discovered module out of `compileModuleImpl()` into its own
separate function and passes it into `compileModuleImpl()` from the
outside. This makes the instance creation logic reusable (useful for my
experiments) and also simplifies the API, removing the `PreBuildStep`
and `PostBuildStep` hooks from `compileModuleImpl()`.
|
|
This PR makes it so that `CompilerInvocation` is the sole owner of the
`PreprocessorOptions` instance.
|
|
function to Triple (#126956)
I'm adding support for SPIR-V, so let's consolidate these checks.
---------
Signed-off-by: Sarnie, Nick <nick.sarnie@intel.com>
|
|
This makes it so that `CompilerInvocation` can be the only entity that
manages ownership of `HeaderSearchOptions`, making it possible to
implement copy-on-write semantics.
|
|
This PR makes the `HeaderSearchOptions` object referenced by
`HeaderSearch` constant. Depends on #130823.
|
|
This uses the systemness of the module map instead of of the Module
instance, as doing otherwise could incorrectly parse the other modules
in that module map as system.
This is still correct as the only ways to get a system module are by the
module map being in a system path, or the module having the [system]
attribute, both of which are handled here.
This makes it so that the systemness of a module is deterministic
instead of depending on the path taken to build it.
|
