| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
mapping
Looks like it breaks win10 builder.
llvm-svn: 312112
|
|
Extend the -fmodule-file option to support the [<name>=]<file> value format.
If the name is omitted, then the old semantics is preserved (the module file
is loaded whether needed or not). If the name is specified, then the mapping
is treated as just another prebuilt module search mechanism, similar to
-fprebuilt-module-path, and the module file is only loaded if actually used
(e.g., via import). With one exception: this mapping also overrides module
file references embedded in other modules (which can be useful if module files
are moved/renamed as often happens during remote compilation).
This override semantics requires some extra work: we now store the module name
in addition to the file name in the serialized AST representation.
Reviewed By: rsmith
Differential Revision: https://reviews.llvm.org/D35020
llvm-svn: 312105
|
|
llvm-svn: 299083
|
|
This reverts commit r298185, effectively reapplying r298165, after fixing the
new unit tests (PR32338). The memory buffer generator doesn't null-terminate
the MemoryBuffer it creates; this version of the commit informs getMemBuffer
about that to avoid the assert.
Original commit message follows:
----
Clang's internal build system for implicit modules uses lock files to
ensure that after a process writes a PCM it will read the same one back
in (without contention from other -cc1 commands). Since PCMs are read
from disk repeatedly while invalidating, building, and importing, the
lock is not released quickly. Furthermore, the LockFileManager is not
robust in every environment. Other -cc1 commands can stall until
timeout (after about eight minutes).
This commit changes the lock file from being necessary for correctness
to a (possibly dubious) performance hack. The remaining benefit is to
reduce duplicate work in competing -cc1 commands which depend on the
same module. Follow-up commits will change the internal build system to
continue after a timeout, and reduce the timeout. Perhaps we should
reconsider blocking at all.
This also fixes a use-after-free, when one part of a compilation
validates a PCM and starts using it, and another tries to swap out the
PCM for something new.
The PCMCache is a new type called MemoryBufferCache, which saves memory
buffers based on their filename. Its ownership is shared by the
CompilerInstance and ModuleManager.
- The ModuleManager stores PCMs there that it loads from disk, never
touching the disk if the cache is hot.
- When modules fail to validate, they're removed from the cache.
- When a CompilerInstance is spawned to build a new module, each
already-loaded PCM is assumed to be valid, and is frozen to avoid
the use-after-free.
- Any newly-built module is written directly to the cache to avoid the
round-trip to the filesystem, making lock files unnecessary for
correctness.
Original patch by Manman Ren; most testcases by Adrian Prantl!
llvm-svn: 298278
|
|
This reverts commit r298165, as it broke the ARM builds.
llvm-svn: 298185
|
|
Clang's internal build system for implicit modules uses lock files to
ensure that after a process writes a PCM it will read the same one back
in (without contention from other -cc1 commands). Since PCMs are read
from disk repeatedly while invalidating, building, and importing, the
lock is not released quickly. Furthermore, the LockFileManager is not
robust in every environment. Other -cc1 commands can stall until
timeout (after about eight minutes).
This commit changes the lock file from being necessary for correctness
to a (possibly dubious) performance hack. The remaining benefit is to
reduce duplicate work in competing -cc1 commands which depend on the
same module. Follow-up commits will change the internal build system to
continue after a timeout, and reduce the timeout. Perhaps we should
reconsider blocking at all.
This also fixes a use-after-free, when one part of a compilation
validates a PCM and starts using it, and another tries to swap out the
PCM for something new.
The PCMCache is a new type called MemoryBufferCache, which saves memory
buffers based on their filename. Its ownership is shared by the
CompilerInstance and ModuleManager.
- The ModuleManager stores PCMs there that it loads from disk, never
touching the disk if the cache is hot.
- When modules fail to validate, they're removed from the cache.
- When a CompilerInstance is spawned to build a new module, each
already-loaded PCM is assumed to be valid, and is frozen to avoid
the use-after-free.
- Any newly-built module is written directly to the cache to avoid the
round-trip to the filesystem, making lock files unnecessary for
correctness.
Original patch by Manman Ren; most testcases by Adrian Prantl!
llvm-svn: 298165
|
|
If we never need to map any ID within the module to its global ID, we don't
need the module offset map. If a compilation transitively depends on lots of
unused module files, this can result in a modest performance improvement.
llvm-svn: 295517
|
|
Oops... r293393 started calling ReadSignature in
ModuleManager::addModule even when there was no ExpectedSignature.
Whether or not this would have a measurable performance impact (I
spotted this by inspection, and ReadSignature should be fairly fast), we
might as well get what we can. Add an extra check against
ExpectedSignature to avoid the hit.
llvm-svn: 293415
|
|
Invert the main branch in ModuleManager::addModule to return early and
reduce indentation, and clean up a bunch of logic as a result. I split
out a function called updateModuleImports to avoid triggering code
duplication.
llvm-svn: 293400
|
|
I don't have a testcase for this (and I'm not sure if it's an observable
bug), but it seems obviously wrong that ModuleManager::removeModules is
failing to clean up deleted modules from ModuleFile::Imports. See the
code in ModuleManager::addModule that inserts into ModuleFile::Imports;
we need the inverse operation.
llvm-svn: 293399
|
|
ModuleManager::removeModules always deletes a tail of the
ModuleManager::Chain. Change the API to enforce that so that we can
simplify the code inside.
There's no real functionality change, although there's a slight
performance hack to loop to the First deleted module instead of the
final module in the chain (skipping the about-to-be-deleted tail).
Also document something suspicious: we fail to clean deleted modules out
of ModuleFile::Imports.
llvm-svn: 293398
|
|
Use std::unique_ptr to clarify the ownership of the ModuleFile instances in
ModuleManager.
llvm-svn: 293395
|
|
Hide the pointer indirection in ModuleManager::begin, ModuleManager::end,
ModuleManager::rbegin, and ModuleManager::rend. Besides tidying up the call
sites, this is preparation for making ownership of ModuleFile explicit.
llvm-svn: 293394
|
|
The main point is to move the delete-the-new-module logic into the same block
that creates it, so I can simplify the memory management in a follow-up, but I
think it's clearer to use use a checkSignature helper here anyway.
There is a minor functionality change: we now scan ahead to pull the signature
out of the control block *only* if this is a new ModuleFile. For old ones,
ASTReader::ReadControlBlock will have already read the signature.
llvm-svn: 293393
|
|
Summary: NFCI
Reviewers: benlangmuir, zturner
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D25948
llvm-svn: 286356
|
|
As proposed on llvm-dev:
http://lists.llvm.org/pipermail/llvm-dev/2016-October/106630.html
Move block info block state to a new class, BitstreamBlockInfo.
Clients may set the block info for a particular cursor with the
BitstreamCursor::setBlockInfo() method.
At this point BitstreamReader is not much more than a container for an
ArrayRef<uint8_t>, so remove it and replace all uses with direct uses
of memory buffers.
Differential Revision: https://reviews.llvm.org/D26259
llvm-svn: 286207
|
|
look for a corresponding file, since we're not going to read it anyway.
No observable behavior change (though we now avoid pointlessly trying to stat
or open a file named "-").
llvm-svn: 280436
|
|
during this function, and to avoid rolling back changes to the module manager's
data structures. Instead, we defer registering the module file until after we
have successfully finished loading it.
llvm-svn: 280434
|
|
This should finish the GraphTraits migration.
Differential Revision: http://reviews.llvm.org/D23730
llvm-svn: 279475
|
|
In this mode, there is no need to load any module map and the programmer can
simply use "@import" syntax to load the module directly from a prebuilt
module path. When loading from prebuilt module path, we don't support
rebuilding of the module files and we ignore compatible configuration
mismatches.
rdar://27290316
Differential Revision: http://reviews.llvm.org/D23125
llvm-svn: 279096
|
|
Summary: Corresponding LLVM patch: D23580
Reviewers: dblaikie
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D23581
llvm-svn: 278963
|
|
Summary: Removed unused headers, replaced some headers with forward class declarations
Patch by: Eugene <claprix@yandex.ru>
Differential Revision: https://reviews.llvm.org/D20100
llvm-svn: 275882
|
|
No functional change is intended, this should just clean things up a
little.
llvm-svn: 273522
|
|
When removing out-of-date modules we might have left behind a VisitOrder
that contains pointers to freed ModuleFiles. This was very rarely seen,
because it only happens when modules go out of date and the VisitOrder
happens to have the right size to not be recomputed.
Thanks ASan!
rdar://23181512
llvm-svn: 250963
|
|
-fmodule-file=.
llvm-svn: 244417
|
|
llvm-svn: 244289
|
|
Avoids the awkward passing of an opaque void *UserData argument. No
functional change intended.
llvm-svn: 243213
|
|
llvm-svn: 242960
|
|
C++ modules. Instead, serialize a list of interesting identifiers and mark those ones out of date on module import. Avoiding the identifier lookups here gives a 20-30% speedup in builds with large numbers of modules. No functionality change intended.
llvm-svn: 242868
|
|
more modules are added: visit modules depth-first rather than breadth-first.
The visitation is still (approximately) oldest-to-newest, and still guarantees
that a module is visited before anything it imports, so modules that are
imported by others sometimes need to jump to a later position in the visitation
order when more modules are loaded, but independent module trees don't
interfere with each other any more.
llvm-svn: 242863
|
|
- introduces a new cc1 option -fmodule-format=[raw,obj]
with 'raw' being the default
- supports arbitrary module container formats that libclang is agnostic to
- adds the format to the module hash to avoid collisions
- splits the old PCHContainerOperations into PCHContainerWriter and
a PCHContainerReader.
Thanks to Richard Smith for reviewing this patch!
llvm-svn: 242499
|
|
llvm-svn: 240353
|
|
The patch is generated using this command:
$ tools/extra/clang-tidy/tool/run-clang-tidy.py -fix \
-checks=-*,llvm-namespace-comment -header-filter='llvm/.*|clang/.*' \
work/llvm/tools/clang
To reduce churn, not touching namespaces spanning less than 10 lines.
llvm-svn: 240270
|
|
A PCHContainerOperations abstract interface provides operations for
creating and unwrapping containers for serialized ASTs (precompiled
headers and clang modules). The default implementation is
RawPCHContainerOperations, which uses a flat file for the output.
The main application for this interface will be an
ObjectFilePCHContainerOperations implementation that uses LLVM to
wrap the module in an ELF/Mach-O/COFF container to store debug info
alongside the AST.
rdar://problem/20091852
llvm-svn: 240225
|
|
Skip imports when we know that we do not need to visit any imports
because we've already deserialized the redecls from a module.
llvm-svn: 237782
|
|
* Strength reduce a std::function to a function pointer,
* Factor out checking the AST file magic number,
* Add a brief doc comment to readAStFileSignature
Thanks to Chandler for spotting these oddities.
llvm-svn: 233050
|
|
consumers of that module.
Previously, such a file would only be available if the module happened to
actually import something from that module.
llvm-svn: 232583
|
|
llvm-svn: 230454
|
|
This is a necessary prerequisite for debugging with modules.
The .pcm files become containers that hold the serialized AST which allows
us to store debug information in the module file that can be shared by all
object files that were built importing the module.
This reapplies r230044 with a fixed configure+make build and updated
dependencies and testcase requirements. Over the last iteration this
version adds
- missing target requirements for testcases that specify an x86 triple,
- a missing clangCodeGen.a dependency to libClang.a in the make build.
rdar://problem/19104245
llvm-svn: 230423
|
|
This reverts commit r230305.
Off to fix another round of missing dependencies on various platforms.
llvm-svn: 230309
|
|
This is a necessary prerequisite for debugging with modules.
The .pcm files become containers that hold the serialized AST which allows
us to store debug information in the module file that can be shared by all
object files that were built importing the module.
rdar://problem/19104245
This reapplies r230044 with a fixed configure+make build and updated
dependencies. Take 3.
llvm-svn: 230305
|
|
This reverts commit 230099.
The Linux configure+make build variant still needs some work.
llvm-svn: 230103
|
|
This is a necessary prerequisite for debugging with modules.
The .pcm files become containers that hold the serialized AST which allows
us to store debug information in the module file that can be shared by all
object files that were built importing the module.
rdar://problem/19104245
This reapplies r230044 with a fixed configure+make build and updated
dependencies. Take 2.
llvm-svn: 230089
|
|
This reverts commit r230067.
Investigating another batch of problems found by the bots.
llvm-svn: 230073
|
|
This is a necessary prerequisite for debugging with modules.
The .pcm files become containers that hold the serialized AST which allows
us to store debug information in the module file that can be shared by all
object files that were built importing the module.
rdar://problem/19104245
This reapplies r230044 with a fixed configure+make build and updated
dependencies.
llvm-svn: 230067
|
|
This reverts commit r230044 while dealing with buildbot breakage.
Conflicts:
test/Modules/module_container.m
llvm-svn: 230052
|
|
This is a necessary prerequisite for debugging with modules.
The .pcm files become containers that hold the serialized AST which allows
us to store debug information in the module file that can be shared by all
object files that were built importing the module.
rdar://problem/19104245
llvm-svn: 230044
|
|
to be newer than we were expecting. That happens if .pcm's get moved between
file systems during a distributed build. (It's still not OK for them to actually
be different, though, so we still check the size and signature matches.)
llvm-svn: 222507
|
|
We may need to verify the signature on subsequent imports as well, just
like we verify the size/modtime:
@import A;
@import B; // imports A
@import C; // imports A
llvm-svn: 221569
|
|
This eliminates converting back and forth between the 3 formats and
gives us a more homogeneous interface.
llvm-svn: 220657
|
