| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
This PR adds new `ModuleCache` interface to Clang's implicitly-built
modules machinery. The main motivation for this change is to create a
second implementation that uses a more efficient kind of
`llvm::AdvisoryLock` during dependency scanning.
In addition to the lock abstraction, the `ModuleCache` interface also
manages the existing `InMemoryModuleCache` instance. I found that
compared to keeping these separate/independent, the code is a bit
simpler now, since these are two tightly coupled concepts. I can
envision a more efficient implementation of the `InMemoryModuleCache`
for the single-process case too, which will be much easier to implement
with the current setup.
This is not intended to be a functional change.
|
|
This PR abstracts the `LockFileManager` API into new `AdvisoryLock`
interface. This is so that we can create an alternative implementation
for Clang implicitly-built modules that is optimized for single-process
environment.
|
|
Clients of `LockFileManager` call `unsafeRemoveLockFile()` whenever
`tryLock()` fails. However looking at the code, there are no scenarios
where this actually does something useful. This PR removes such calls.
|
|
This patch removes some internal state out of `LockFileManager` by
moving the locking code from the constructor into new member function
`tryLock()` which returns the errors right away. This simplifies and
modernizes the interface.
|
|
Currently, `LockFileManager` assumes the owner of the lock file creates
an output file. This is problematic for at least three reasons:
1. It is orthogonal to the main purpose of this class - mutual
exclusion. This makes creating an alternative implementation more
complicated than it needs to be.
2. Some clients (like the upstream `AMDGPUSplitModule.cpp` file) assume
the output file is not necessary. The owner of the lock file does not
write the file expected by `LockFileManager` and the processes waiting
for the non-owned lock file to be unlocked therefore assume the owner
has died. This means that the work gets repeated by each waiting
process, serially.
3. The documentation makes it sound like successfully waiting for a
non-owned lock file guarantees the output file to be present on the file
system. Implicitly-built modules rely on this. However, the module file
may disappear between `LockFileManager` performing the check and the
compiler loading the module (for example due to module cache pruning
with short intervals, or intervention from outside of Clang). The
compiler assumes this cannot happen, and fails the build if it does.
This PR solves this situation by removing the check, reflecting that in
the `LockFileManager` documentation, and fixing the time-of-check
time-of-use bug in implicit modules.
|
|
... to improve consistency and make "Clang time report" cover
`FrontendAction::BeginSourceFile` and `FrontendAction::EndSourceFile`.
|
|
The code generation time is unclear in the -ftime-report output:
* The two clang timers "Code Generation Time" and "LLVM IR Generation
Time" are in the default group "Miscellaneous Ungrouped Timers".
* There is also a "Clang front-end time" group, which actually includes
code generation time.
```
===-------------------------------------------------------------------------===
Miscellaneous Ungrouped Timers
===-------------------------------------------------------------------------===
---User Time--- --System Time-- --User+System-- ---Wall Time--- --- Name ---
0.0611 ( 1.7%) 0.0099 ( 4.4%) 0.0710 ( 1.9%) 0.0713 ( 1.9%) LLVM IR Generation Time
3.5140 ( 98.3%) 0.2165 ( 95.6%) 3.7306 ( 98.1%) 3.7342 ( 98.1%) Code Generation Time
3.5751 (100.0%) 0.2265 (100.0%) 3.8016 (100.0%) 3.8055 (100.0%) Total
...
===-------------------------------------------------------------------------===
Clang front-end time report
===-------------------------------------------------------------------------===
Total Execution Time: 3.9108 seconds (3.9146 wall clock)
---User Time--- --System Time-- --User+System-- ---Wall Time--- --- Name ---
3.6802 (100.0%) 0.2306 (100.0%) 3.9108 (100.0%) 3.9146 (100.0%) Clang front-end timer
3.6802 (100.0%) 0.2306 (100.0%) 3.9108 (100.0%) 3.9146 (100.0%) Total
```
This patch
* renames "Clang front-end time report" (FrontendAction time) to "Clang
time report",
* renames "Clang front-end" to "Front end",
* moves "LLVM IR Generation" into the group,
* replaces "Code Generation time" with "Optimizer" (middle end) and
"Machine code generation" (back end).
```
% clang -c sqlite3.i -w -ftime-report -mllvm -sort-timers=0
...
===-------------------------------------------------------------------------===
Clang time report
===-------------------------------------------------------------------------===
Total Execution Time: 1.5922 seconds (1.5972 wall clock)
---User Time--- --System Time-- --User+System-- ---Wall Time--- --- Name ---
0.5107 ( 35.9%) 0.0105 ( 6.2%) 0.5211 ( 32.7%) 0.5222 ( 32.7%) Front end
0.2464 ( 17.3%) 0.0340 ( 20.0%) 0.2804 ( 17.6%) 0.2814 ( 17.6%) LLVM IR generation
0.6240 ( 43.9%) 0.1235 ( 72.7%) 0.7475 ( 47.0%) 0.7503 ( 47.0%) Machine code generation
0.0413 ( 2.9%) 0.0018 ( 1.0%) 0.0431 ( 2.7%) 0.0433 ( 2.7%) Optimizer
1.4224 (100.0%) 0.1698 (100.0%) 1.5922 (100.0%) 1.5972 (100.0%) Total
```
Pull Request: https://github.com/llvm/llvm-project/pull/122225
|
|
This reverts commit a1153cd6fedd4c906a9840987934ca4712e34cb2 with fixes
to lldb breakages.
Fixes https://github.com/llvm/llvm-project/issues/117145.
|
|
Reverted for causing:
https://github.com/llvm/llvm-project/issues/117145
This reverts commit bdd10d9d249bd1c2a45e3de56a5accd97e953458.
|
|
Starting with 41e3919ded78d8870f7c95e9181c7f7e29aa3cc4 DiagnosticsEngine
creation might perform IO. It was implicitly defaulting to
getRealFileSystem. This patch makes it explicit by pushing the decision
making to callers.
It uses ambient VFS if one is available, and keeps using
`getRealFileSystem` if there aren't any VFS.
|
|
This reverts commit 5f140ba54794fe6ca379362b133eb27780e363d7.
|
|
This reverts commit 12e3ed8de8c6063b15916b3faf67c8c9cd17df1f.
This reverts commit 41e3919ded78d8870f7c95e9181c7f7e29aa3cc4.
There are some buildbot breakages in
https://lab.llvm.org/buildbot/#/builders/18/builds/6832.
|
|
This implements
https://discourse.llvm.org/t/rfc-add-support-for-controlling-diagnostics-severities-at-file-level-granularity-through-command-line/81292.
Users now can suppress warnings for certain headers by providing a
mapping with globs, a sample file looks like:
```
[unused]
src:*
src:*clang/*=emit
```
This will suppress warnings from `-Wunused` group in all files that
aren't under `clang/` directory. This mapping file can be passed to
clang via `--warning-suppression-mappings=foo.txt`.
At a high level, mapping file is stored in DiagnosticOptions and then
processed with rest of the warning flags when creating a
DiagnosticsEngine. This is a functor that uses SpecialCaseLists
underneath to match against globs coming from the mappings file.
This implies processing warning options now performs IO, relevant
interfaces are updated to take in a VFS, falling back to RealFileSystem
when one is not available.
|
|
Some `FileManager` APIs still return `{File,Directory}Entry` instead of
the preferred `{File,Directory}EntryRef`. These are documented to be
deprecated, but don't have the attribute that warns on their usage. This
PR marks them as such with `LLVM_DEPRECATED()` and replaces their usage
with the recommended counterparts. NFCI.
|
|
As specified in the docs,
1) raw_string_ostream is always unbuffered and
2) the underlying buffer may be used directly
( 65b13610a5226b84889b923bae884ba395ad084d for further reference )
* Don't call raw_string_ostream::flush(), which is essentially a no-op.
* Avoid unneeded calls to raw_string_ostream::str(), to avoid excess indirection.
|
|
Clang's `-cc1 -print-stats` shows lots of useful internal data including
basic `FileManager` stats. Since this layer caches some results, it is
unclear how that information translates to actual filesystem accesses.
This PR uses `llvm::vfs::TracingFileSystem` to provide that missing
information.
Similar mechanism is implemented for `clang-scan-deps`'s verbose mode
(`-v`). IO contention proved to be a real bottleneck a couple of times
already and this new feature should make those easier to detect in the
future. The tracing VFS is inserted below the caching FS and above the
real FS.
|
|
This patch stops adjustments of the module cache path beyond what is
done in `ParseHeaderSearchArgs` (making it absolute and removing dots).
This enables more efficient implementation of the caching VFS in
https://github.com/llvm/llvm-project/pull/88800.
|
|
|
|
code paths in `FileManager`. (#88427)
`FileManager::getDirectoryRef()` and `FileManager::getFileRef()` are hot code paths in `clang-scan-deps`. These functions are updating on every call a few atomics related to printing statistics, which causes contention on high core count machines.
After this patch we make the variables local to the `FileManager`.
In our test case, this saves about 49 sec over 1 min 47 sec of `clang-scan-deps` run time (1 min 47 sec before, 58 sec after). These figures are after applying my suggestion in https://github.com/llvm/llvm-project/pull/88152#issuecomment-2049803229, that is:
```
static bool shouldCacheStatFailures(StringRef Filename) {
return true;
}
```
Without the above, there's just too much OS noise from the high volume of `status()` calls with regular non-modules C++ code. Tested on Windows with clang-cl.
|
|
An instance of `PreprocessorOptions` is part of `CompilerInvocation`
which is supposed to be a value type. The `FailedModules` member is
problematic, since it's essentially a shared state used by multiple
`CompilerInstance` objects, and not really a preprocessor option. Let's
move it into `CompilerInstance` instead.
|
|
(#86216)
The `ASTWriter` algorithm for computing affecting module maps uses
`SourceManager::translateFile()` to get a `FileID` from a `FileEntry`.
This is slow (O(n)) since the function performs a linear walk over
`SLocEntries` until it finds one with a matching `FileEntry`.
This patch removes this use of `SourceManager::translateFile()` by
tracking `FileID` instead of `FileEntry` in couple of places in
`ModuleMap`, giving `ASTWriter` the desired `FileID` directly. There are
no changes required for clients that still want a `FileEntry` from
`ModuleMap`: the existing APIs internally use `SourceManager` to perform
the reverse `FileID` to `FileEntry` conversion in O(1).
|
|
This patch enables support that the XL compiler had for AIX under
-qdatalocal/-qdataimported.
|
|
Before this patch, if a module fails to build because of a missing
config_macro, the user will never see the config macro warning. This
patch diagnoses this before building, and each subsequent time a module
is imported.
rdar://123921931
|
|
|
|
https://github.com/llvm/llvm-project/pull/76471 caused buildbot failures
on Windows. For more details, see
https://github.com/llvm/llvm-project/issues/77546.
This PR revises the test and relands
https://github.com/llvm/llvm-project/pull/76471.
|
|
|
|
|
|
Issue #77546
This reverts commit 07c9189fcc063bdf6219d2733843c89cde3991e1.
|
|
This PR exposes four PGO functions
- `__llvm_profile_set_filename`
- `__llvm_profile_reset_counters`,
- `__llvm_profile_dump`
- `__llvm_orderfile_dump`
to user programs through the new header `instr_prof_interface.h` under
`compiler-rt/include/profile`. This way, the user can include the header
`profile/instr_prof_interface.h` to introduce these four names to their
programs.
Additionally, this PR defines macro `__LLVM_INSTR_PROFILE_GENERATE` when
the program is compiled with profile generation, and defines macro
`__LLVM_INSTR_PROFILE_USE` when the program is compiled with profile
use. `__LLVM_INSTR_PROFILE_GENERATE` together with
`instr_prof_interface.h` define the PGO functions only when the program
is compiled with profile generation. When profile generation is off,
these PGO functions are defined away and leave no trace in the user's
program.
Background:
https://discourse.llvm.org/t/pgo-are-the-llvm-profile-functions-stable-c-apis-across-llvm-releases/75832
|
|
|
|
This upstreams more of the Clang API Notes functionality that is
currently implemented in the Apple fork:
https://github.com/apple/llvm-project/tree/next/clang/lib/APINotes
|
|
This upstreams more of the Clang API Notes functionality that is
currently implemented in the Apple fork:
https://github.com/apple/llvm-project/tree/next/clang/lib/APINotes
This adds the initial Clang APINotes infrastructure to Clang Sema and
Clang Frontend.
There will shortly be a follow-up patch with the actual usages of this
API. I'm splitting these changes into separate PRs to keep the diffs
easier to review.
|
|
instead of pointer (#67902)
The `Module` parameter to `checkModuleIsAvailable` is currently passed
by pointer to non-const. However, it requires only const access and it
cannot be null. Change this to be a reference to const instead.
This then makes it obvious that it is an input-only parameter, so move
it to be before the in-out parameter for diagnostics.
|
|
|
|
The hook introduced by https://reviews.llvm.org/D22815 for AMDGPU was
removed by commit ce2258c1cd5dc9cf20040d1b1e540d80250c1435 in 2020.
|
|
for C++20 modules
Previous implementation of the warning to use `-fmodule-file=<BMIPath>`
for C++20 Modules is not straightforward and it is problematic in case
we read the BMIPath by writing tools based clang components. This patch
refactors it with a simple and direct style.
|
|
|
|
Add a way to enable -Wsystem-headers only for a specific module. This is
useful for validating a module that would otherwise not see system
header diagnostics without being flooded by diagnostics for unrelated
headers/modules. It's relatively common for a module to be marked
[system] but still wish to validate itself explicitly.
rdar://113401565
Differential Revision: https://reviews.llvm.org/D156948
|
|
temp files
Additional patch to https://reviews.llvm.org/D103806 to add the same flags in the path where the CreateMissingDirectories booleans is set to true.
Reviewed By: abhina.sreeskantharajan
Differential Revision: https://reviews.llvm.org/D155651
|
|
This patch renames the `OpenMPIRBuilderConfig` flags to reduce confusion over
their meaning. `IsTargetCodegen` becomes `IsGPU`, whereas `IsEmbedded` becomes
`IsTargetDevice`. The `-fopenmp-is-device` compiler option is also renamed to
`-fopenmp-is-target-device` and the `omp.is_device` MLIR attribute is renamed
to `omp.is_target_device`. Getters and setters of all these renamed properties
are also updated accordingly. Many unit tests have been updated to use the new
names, but an alias for the `-fopenmp-is-device` option is created so that
external programs do not stop working after the name change.
`IsGPU` is set when the target triple is AMDGCN or NVIDIA PTX, and it is only
valid if `IsTargetDevice` is specified as well. `IsTargetDevice` is set by the
`-fopenmp-is-target-device` compiler frontend option, which is only added to
the OpenMP device invocation for offloading-enabled programs.
Differential Revision: https://reviews.llvm.org/D154591
|
|
In file `clang/lib/Basic/Module.cpp` the `Module` class had `submodule_begin()` and `submodule_end()` functions to retrieve corresponding iterators for private vector of Modules. This commit removes mentioned functions, and replaces all of theirs usages with `submodules()` function and range-based for-loops.
Differential Revision: https://reviews.llvm.org/D148954
|
|
While we eventually want to remove the mapping from .Private to _Private
modules, until we do, ensure that it behaves the same for explicit
modules.
rdar://107449872
Differential Revision: https://reviews.llvm.org/D147477
|
|
in order to provide as much information as possible to the user. The
diagnostic will now look like for example:
error: error reading '/tmp/foo.c': Permission denied
(This addresses a FIXME from 2019, 9ef6c49baf45)
Differential revision: https://reviews.llvm.org/D146280
|
|
environment variable.
Using two environment variables `CC_PRINT_INTERNAL_STAT` and
`CC_PRINT_INTERNAL_STAT_FILE` to work like `CC_PRINT_PROC_STAT`.
The purpose of the change is to allow collecting the internal stats
without modifying the build scripts. Write all stats to a single file
to simplify aggregating the data.
Differential Revision: https://reviews.llvm.org/D144981
|
|
The forwarding header is left in place because of its use in
`polly/lib/External/isl/interface/extract_interface.cc`, but I have
added a GCC warning about the fact it is deprecated, because it is used
in `isl` from where it is included by Polly.
|
|
implementation unit and document the behavior
Close https://github.com/llvm/llvm-project/issues/57293.
Previsouly we can't use `-fmodule-file=<module-name>=<BMI-Path>` for
implementation units, it is a bug. Also the behavior of the above option
is not tested nor documented for C++20 Modules. This patch addresses the
2 problems.
|
|
Close https://github.com/llvm/llvm-project/issues/60544.
The root cause for the issue is that when we compile a module unit, the
file manager (and proprocessor and source manager) are owned by AST
instead of the compilaton instance. So the file manager may be invalid
when we want to create a time-report file for -ftime-trace when we are
compiling a module unit.
This patch tries to recreate the file manager for -ftime-trace if we
find the file manager is not valid.
|
|
This patch replaces (llvm::|)Optional< with std::optional<. I'll post
a separate patch to remove #include "llvm/ADT/Optional.h".
This is part of an effort to migrate from llvm::Optional to
std::optional:
https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
|
|
This reverts commit d20101db48945e9d7a19ce3edcfd91d7e1aeadab.
Lifetime of the string is not what I thought it was it seems.
|
|
PathStorage is only used in one of the if branches, so doesn't need to
be a std::optional anyway.
|
