aboutsummaryrefslogtreecommitdiff
path: root/clang/lib/Basic/VirtualFileSystem.cpp
AgeCommit message (Collapse)AuthorFilesLines
2016-05-12Revert "[VFS] Reapply r269100: Reconstruct the VFS overlay tree for more ↵Bruno Cardoso Lopes1-80/+1
accurate lookup" Reverts r269270, buildbots still failing: http://lab.llvm.org:8011/builders/clang-x64-ninja-win7/builds/12119 http://bb.pgr.jp/builders/ninja-clang-i686-msc19-R/builds/2847 llvm-svn: 269276
2016-05-12[VFS] Reapply r269100: Reconstruct the VFS overlay tree for more accurate lookupBruno Cardoso Lopes1-1/+80
The way we currently build the internal VFS overlay representation leads to inefficient path search and might yield wrong answers when asked for recursive or regular directory iteration. Currently, when reading an YAML file, each YAML root entry is placed inside a new root in the filesystem overlay. In the crash reproducer, a simple "@import Foundation" currently maps to 43 roots, and when looking up paths, we traverse a directory tree for each of these different roots, until we find a match (or don't). This has two consequences: - It's slow. - Directory iteration gives incomplete results since it only return results within one root - since contents of the same directory can be declared inside different roots, the result isn't accurate. This is in part fault of the way we currently write out the YAML file when emitting the crash reproducer - we could generate only one root and that would make it fast and correct again. However, we should not rely on how the client writes the YAML, but provide a good internal representation regardless. This patch builds a proper virtual directory tree out of the YAML representation, allowing faster search and proper iteration. Besides the crash reproducer, this potentially benefits other VFS clients. llvm-svn: 269270
2016-05-11Hopefully bring llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast back to lifeSean Silva1-80/+1
Bruno made a couple valiant attempts but the bot is still red. This reverts r269100 (primary commit), r269108 (fix attempt), r269133 (fix attempt). llvm-svn: 269160
2016-05-10[VFS] Reconstruct the VFS overlay tree for more accurate lookupBruno Cardoso Lopes1-1/+80
The way we currently build the internal VFS overlay representation leads to inefficient path search and might yield wrong answers when asked for recursive or regular directory iteration. Currently, when reading an YAML file, each YAML root entry is placed inside a new root in the filesystem overlay. In the crash reproducer, a simple "@import Foundation" currently maps to 43 roots, and when looking up paths, we traverse a directory tree for each of these different roots, until we find a match (or don't). This has two consequences: - It's slow. - Directory iteration gives incomplete results since it only return results within one root - since contents of the same directory can be declared inside different roots, the result isn't accurate. This is in part fault of the way we currently write out the YAML file when emitting the crash reproducer - we could generate only one root and that would make it fast and correct again. However, we should not rely on how the client writes the YAML, but provide a good internal representation regardless. This patch builds a proper virtual directory tree out of the YAML representation, allowing faster search and proper iteration. Besides the crash reproducer, this potentially benefits other VFS clients. llvm-svn: 269100
2016-05-06[VFS] Add dump methods to the VFS overlay treeBruno Cardoso Lopes1-0/+24
Useful when debugging issues within the VFS overlay. llvm-svn: 268820
2016-04-13[CrashReproducer] Setup 'use-external-names' in YAML files.Bruno Cardoso Lopes1-4/+9
Hide the real paths when rebuilding from VFS by setting up the crash reproducer to use 'use-external-names' = false. This way we avoid module redifinition errors and consistently use the same paths against all modules. With this change on Darwin we are able to simulate a crash for a simple application using "Foundation/Foundation.h" (which relies on a bunch of different frameworks and headers) and successfully rebuild all the modules by relying solely at the VFS overlay. llvm-svn: 266234
2016-04-13[VFS] Move default values to in-class member initialization. NFCBruno Cardoso Lopes1-3/+3
llvm-svn: 266233
2016-03-30[VFS] Handle empty entries in directory traversalBruno Cardoso Lopes1-8/+13
The VFS YAML files contain empty directory entries to describe that it's returning from a subdirectory before describing new files in the parent. In the future, we should properly sort and write YAML files avoiding such empty dirs and mitigate the extra recurson cost. However, since this is used by previous existing YAMLs, make the traversal work in their presence. rdar://problem/24499339 llvm-svn: 264970
2016-03-26Check if a path is already absolute before trying to make it so.Bob Wilson1-0/+3
The FileSystem::makeAbsolute function has been calculating the current working directory unconditionally, even when it is not needed. This calls down to llvm::sys::fs::current_path, which is relatively expensive because it stats two directories, regardless of whether those paths are already in the stat cache. The net effect is that when using the VFS, every stat during header search turns into three stats. With this change, we get back to a single stat for absolute directory paths. llvm-svn: 264519
2016-03-20Reapply [2] [VFS] Add 'overlay-relative' field to YAML filesBruno Cardoso Lopes1-18/+95
This reapplies r261552 and r263748. Fixed testcase to reapply. The VFS overlay mapping between virtual paths and real paths is done through the 'external-contents' entries in YAML files, which contains hardcoded paths to the real files. When a module compilation crashes, headers are dumped into <name>.cache/vfs directory and are mapped via the <name>.cache/vfs/vfs.yaml. The script generated for reproduction uses -ivfsoverlay pointing to file to gather the mapping between virtual paths and files inside <name>.cache/vfs. Currently, we are only capable of reproducing such crashes in the same machine as they happen, because of the hardcoded paths in 'external-contents'. To be able to reproduce a crash in another machine, this patch introduces a new option in the VFS yaml file called 'overlay-relative'. When it's equal to 'true' it means that the provided path to the YAML file through the -ivfsoverlay option should also be used to prefix the final path for every 'external-contents'. Example, given the invocation snippet "... -ivfsoverlay <name>.cache/vfs/vfs.yaml" and the following entry in the yaml file: "overlay-relative": "true", "roots": [ ... "type": "directory", "name": "/usr/include", "contents": [ { "type": "file", "name": "stdio.h", "external-contents": "/usr/include/stdio.h" }, ... Here, a file manager request for virtual "/usr/include/stdio.h", that will map into real path "/<absolute_path_to>/<name>.cache/vfs/usr/include/stdio.h. This is a useful feature for debugging module crashes in machines other than the one where the error happened. Differential Revision: http://reviews.llvm.org/D17457 rdar://problem/24499339 llvm-svn: 263893
2016-03-17Revert "Reapply [VFS] Add 'overlay-relative' field to YAML files"Bruno Cardoso Lopes1-95/+18
Tests failing on http://bb.pgr.jp/builders/cmake-clang-x86_64-linux/builds/46102 This reverts commit a1683cd6c9e07359c09f86e98a4db6b4e1bc51fc. llvm-svn: 263750
2016-03-17Reapply [VFS] Add 'overlay-relative' field to YAML filesBruno Cardoso Lopes1-18/+95
This reapplies r261552. The VFS overlay mapping between virtual paths and real paths is done through the 'external-contents' entries in YAML files, which contains hardcoded paths to the real files. When a module compilation crashes, headers are dumped into <name>.cache/vfs directory and are mapped via the <name>.cache/vfs/vfs.yaml. The script generated for reproduction uses -ivfsoverlay pointing to file to gather the mapping between virtual paths and files inside <name>.cache/vfs. Currently, we are only capable of reproducing such crashes in the same machine as they happen, because of the hardcoded paths in 'external-contents'. To be able to reproduce a crash in another machine, this patch introduces a new option in the VFS yaml file called 'overlay-relative'. When it's equal to 'true' it means that the provided path to the YAML file through the -ivfsoverlay option should also be used to prefix the final path for every 'external-contents'. Example, given the invocation snippet "... -ivfsoverlay <name>.cache/vfs/vfs.yaml" and the following entry in the yaml file: "overlay-relative": "true", "roots": [ ... "type": "directory", "name": "/usr/include", "contents": [ { "type": "file", "name": "stdio.h", "external-contents": "/usr/include/stdio.h" }, ... Here, a file manager request for virtual "/usr/include/stdio.h", that will map into real path "/<absolute_path_to>/<name>.cache/vfs/usr/include/stdio.h. This is a useful feature for debugging module crashes in machines other than the one where the error happened. Differential Revision: http://reviews.llvm.org/D17457 rdar://problem/24499339 llvm-svn: 263748
2016-03-17Reapply [2]: [VFS] Add support for handling path traversalsBruno Cardoso Lopes1-16/+64
This was applied twice r261551 and 263617 and later reverted because: (1) Windows bot failing on unittests. Change the current behavior to do not handle path traversals on windows. (2) Windows bot failed to include llvm/Config/config.h in order to use HAVE_REALPATH. Use LLVM_ON_UNIX instead, as done in lib/Basic/FileManager.cpp. Handle ".", ".." and "./" with trailing slashes while collecting files to be dumped into the vfs overlay directory. Include the support for symlinks into components. Given the path: /install-dir/bin/../lib/clang/3.8.0/include/altivec.h, if "bin" component is a symlink, it's not safe to use `path::remove_dots` here, and `realpath` is used to get the right answer. Since `realpath` is expensive, we only do it at collecting time (which only happens during the crash reproducer) and cache the base directory for fast lookups. Overall, this makes the input to the VFS YAML file to be canonicalized to never contain traversal components. Differential Revision: http://reviews.llvm.org/D17104 rdar://problem/24499339 llvm-svn: 263686
2016-03-16Revert r263617, "Reapply: [VFS] Add support for handling path traversals"NAKAMURA Takumi1-64/+16
It broke standalone clang build. llvm-svn: 263636
2016-03-16Reapply: [VFS] Add support for handling path traversalsBruno Cardoso Lopes1-16/+64
This is originally r261551, reverted because of windows bots failing on unittests. Change the current behavior to do not handle path traversals on windows. Handle ".", ".." and "./" with trailing slashes while collecting files to be dumped into the vfs overlay directory. Include the support for symlinks into components. Given the path: /install-dir/bin/../lib/clang/3.8.0/include/altivec.h, if "bin" component is a symlink, it's not safe to use `path::remove_dots` here, and `realpath` is used to get the right answer. Since `realpath` is expensive, we only do it at collecting time (which only happens during the crash reproducer) and cache the base directory for fast lookups. Overall, this makes the input to the VFS YAML file to be canonicalized to never contain traversal components. Differential Revision: http://reviews.llvm.org/D17104 rdar://problem/24499339 llvm-svn: 263617
2016-03-04[VFS] Switch from close to SafelyCloseFileDescriptorDavid Majnemer1-13/+3
The SafelyCloseFileDescriptor machinery does the right thing in the face of signals while close will do something platform specific which results in the FD potentially getting leaked. llvm-svn: 262687
2016-02-23Revert "[VFS] Add support for handling path traversals"Bruno Cardoso Lopes1-39/+15
This reverts commit r261551 due to failing tests in windows bots: http://lab.llvm.org:8011/builders/clang-x64-ninja-win7/builds/10054 Failing Tests (4): Clang-Unit :: Basic/BasicTests.exe/VFSFromYAMLTest.CaseInsensitive Clang-Unit :: Basic/BasicTests.exe/VFSFromYAMLTest.DirectoryIteration Clang-Unit :: Basic/BasicTests.exe/VFSFromYAMLTest.MappedFiles Clang-Unit :: Basic/BasicTests.exe/VFSFromYAMLTest.UseExternalName llvm-svn: 261654
2016-02-23Revert "[VFS] Add 'overlay-relative' field to YAML files" and "[VFS] Fix ↵Bruno Cardoso Lopes1-98/+19
call to getVFSFromYAML in unittests" This reverts commit r261552 and r261556 because of failing unittests on windows: Failing Tests (4): Clang-Unit :: Basic/BasicTests.exe/VFSFromYAMLTest.CaseInsensitive Clang-Unit :: Basic/BasicTests.exe/VFSFromYAMLTest.DirectoryIteration Clang-Unit :: Basic/BasicTests.exe/VFSFromYAMLTest.MappedFiles Clang-Unit :: Basic/BasicTests.exe/VFSFromYAMLTest.UseExternalName llvm-svn: 261613
2016-02-22[VFS] Add 'overlay-relative' field to YAML filesBruno Cardoso Lopes1-19/+98
The VFS overlay mapping between virtual paths and real paths is done through the 'external-contents' entries in YAML files, which contains hardcoded paths to the real files. When a module compilation crashes, headers are dumped into <name>.cache/vfs directory and are mapped via the <name>.cache/vfs/vfs.yaml. The script generated for reproduction uses -ivfsoverlay pointing to file to gather the mapping between virtual paths and files inside <name>.cache/vfs. Currently, we are only capable of reproducing such crashes in the same machine as they happen, because of the hardcoded paths in 'external-contents'. To be able to reproduce a crash in another machine, this patch introduces a new option in the VFS yaml file called 'overlay-relative'. When it's equal to 'true' it means that the provided path to the YAML file through the -ivfsoverlay option should also be used to prefix the final path for every 'external-contents'. Example, given the invocation snippet "... -ivfsoverlay <name>.cache/vfs/vfs.yaml" and the following entry in the yaml file: "overlay-relative": "true", "roots": [ ... "type": "directory", "name": "/usr/include", "contents": [ { "type": "file", "name": "stdio.h", "external-contents": "/usr/include/stdio.h" }, ... Here, a file manager request for virtual "/usr/include/stdio.h", that will map into real path "/<absolute_path_to>/<name>.cache/vfs/usr/include/stdio.h. This is a useful feature for debugging module crashes in machines other than the one where the error happened. Differential Revision: http://reviews.llvm.org/D17457 rdar://problem/24499339 llvm-svn: 261552
2016-02-22[VFS] Add support for handling path traversalsBruno Cardoso Lopes1-15/+39
Handle ".", ".." and "./" with trailing slashes while collecting files to be dumped into the vfs overlay directory. Include the support for symlinks into components. Given the path: /install-dir/bin/../lib/clang/3.8.0/include/altivec.h, if "bin" component is a symlink, it's not safe to use `path::remove_dots` here, and `realpath` is used to get the right answer. Since `realpath` is expensive, we only do it at collecting time (which only happens during the crash reproducer) and cache the base directory for fast lookups. Overall, this makes the input to the VFS YAML file to be canonicalized to never contain traversal components. Differential Revision: http://reviews.llvm.org/D17104 rdar://problem/24499339 llvm-svn: 261551
2016-01-09[vfs] Normalize working directory if requested.Benjamin Kramer1-0/+17
FixedCompilationDatabase sets the working dir to "." by default. For chdir(".") this is a noop but this lead to InMemoryFileSystem to create bogus paths. Fixes PR25327. llvm-svn: 257260
2015-12-10[VFS] Fix status() of opened redirected fileBen Langmuir1-22/+28
Make RedirectedFileSystem::openFilForRead(path)->status() the same as RedirectedFileSystem::status(path). Previously we would just get the status of the underlying real file, which would not have the IsVFSMapped bit set. This fixes rebuilding a module that has an include that is relative to the includer where we will lookup the real path of that file before we lookup the VFS location. rdar://problem/23640339 llvm-svn: 255312
2015-11-30Use std::begin() and std::end() instead of doing the same manually. NFCCraig Topper1-4/+2
llvm-svn: 254281
2015-11-09Moving FileManager::removeDotPaths to llvm::sys::path::remove_dotsMike Aizatsky1-2/+2
Differential Revision: http://reviews.llvm.org/D14394 llvm-svn: 252501
2015-10-20Roll-back r250822.Angel Garcia Gomez1-5/+5
Summary: It breaks the build for the ASTMatchers Subscribers: klimek, cfe-commits Differential Revision: http://reviews.llvm.org/D13893 llvm-svn: 250827
2015-10-20Apply modernize-use-default to clang.Angel Garcia Gomez1-5/+5
Summary: Replace empty bodies of default constructors and destructors with '= default'. Reviewers: bkramer, klimek Subscribers: klimek, alexfh, cfe-commits Differential Revision: http://reviews.llvm.org/D13890 llvm-svn: 250822
2015-10-12[VFS] Let the user decide if they want path normalization.Benjamin Kramer1-16/+30
This is a more principled version of what I did earlier. Path normalization is generally a good thing, but may break users in strange environments, e. g. using lots of symlinks. Let the user choose and default it to on. This also changes adding a duplicated file into returning an error if the file contents are different instead of an assertion failure. Differential Revision: http://reviews.llvm.org/D13658 llvm-svn: 250060
2015-10-12[VFS] Don't try to be heroic with '.' in paths.Benjamin Kramer1-8/+11
Actually the only special path we have to handle is ./foo, the rest is tricky to get right so do the same thing as the existing YAML vfs here. llvm-svn: 250036
2015-10-12[VFS] remove handling of '..' for now.Benjamin Kramer1-2/+2
This can fail badly if we're overlaying a real file system and there are symlinks there. Just keep the path as-is for now. This essentially reverts r249830. llvm-svn: 250021
2015-10-09[VFS] Rename RedirectingFS internals to avoid collisions with public clang ↵Benjamin Kramer1-24/+29
classes Hopefully fixes the MSVC build. NFC intended. llvm-svn: 249832
2015-10-09[VFS] Just normalize away .. and . in paths for in-memory file systems.Benjamin Kramer1-17/+10
This simplifies the code and gets us support for .. for free. llvm-svn: 249830
2015-10-07[VFS] Port driver tool chains to VFS.Benjamin Kramer1-0/+5
There are still some loose ends here but it's sufficient so we can detect GCC headers that are inside of a VFS. llvm-svn: 249556
2015-10-07[VFS] Refactor VFSFromYAML a bit.Benjamin Kramer1-60/+57
- Rename it to RedirectingFileSystem. This is what it does, YAML is just a serialization format for it. - Consistently use unique_ptr for memory management. No functional change intended. llvm-svn: 249532
2015-10-07[VFS] Also drop '.' when adding files to an in-memory FS.Benjamin Kramer1-0/+7
Otherwise we won't be able to find them later. llvm-svn: 249525
2015-10-06[VFS] Put the incoming name in the file status to make InMemoryFS behave ↵Benjamin Kramer1-1/+1
more like a real FS. llvm-svn: 249409
2015-10-06[VFS] Transition clang-format to use an in-memory FS.Benjamin Kramer1-0/+16
Apart from being cleaner this also means that clang-format no longer has access to the host file system. This isn't necessary because clang-format never reads includes :) Includes minor tweaks and bugfixes found in the VFS implementation while running clang-format tests. llvm-svn: 249385
2015-10-05Remove duplicated default arguments. NFC.Benjamin Kramer1-10/+8
llvm-svn: 249355
2015-10-05[VFS] Fix the windows build by including the right headers.Benjamin Kramer1-0/+8
llvm-svn: 249319
2015-10-05[VFS] Fix compilation on systems where time_t is not int64_t.Benjamin Kramer1-4/+4
No functional change intended. llvm-svn: 249318
2015-10-05[VFS] Add working directories to every virtual file system.Benjamin Kramer1-8/+65
For RealFileSystem this is getcwd()/chdir(), the synthetic file systems can make up one for themselves. OverlayFileSystem now synchronizes the working directories when a new FS is added to the overlay or the overlay working directory is set. This allows purely artificial file systems that have zero ties to the underlying disks. Differential Revision: http://reviews.llvm.org/D13430 llvm-svn: 249316
2015-10-05[VFS] Add an in-memory file system implementation.Benjamin Kramer1-0/+246
This is a simple file system tree of memory buffers that can be filled by a client. In conjunction with an OverlayFS it can be used to make virtual files accessible right next to physical files. This can be used as a replacement for the virtual file handling in FileManager and which I intend to remove eventually. llvm-svn: 249315
2015-10-05[VFS] Move class out of method so it looks less like Java.Benjamin Kramer1-26/+28
No functionality change. llvm-svn: 249314
2015-10-05[VFS] Remove setName from the file interface.Benjamin Kramer1-36/+62
This streamlines the interface a bit and makes Status more immutable. No functional change intended. llvm-svn: 249310
2015-06-24Remove a limited and somewhat questionable DenseMapInfo specializationChandler Carruth1-14/+0
for StringRef now that the core DenseMap library provides this facility. llvm-svn: 240530
2015-06-22Revert r240270 ("Fixed/added namespace ending comments using clang-tidy").Alexander Kornienko1-2/+2
llvm-svn: 240353
2015-06-22Fixed/added namespace ending comments using clang-tidy. NFCAlexander Kornienko1-2/+2
The patch is generated using this command: $ tools/extra/clang-tidy/tool/run-clang-tidy.py -fix \ -checks=-*,llvm-namespace-comment -header-filter='llvm/.*|clang/.*' \ work/llvm/tools/clang To reduce churn, not touching namespaces spanning less than 10 lines. llvm-svn: 240270
2015-04-11Use 'override/final' instead of 'virtual' for overridden methodsAlexander Kornienko1-3/+3
Summary: The patch is generated using clang-tidy misc-use-override check. This command was used: tools/clang/tools/extra/clang-tidy/tool/run-clang-tidy.py \ -checks='-*,misc-use-override' -header-filter='llvm|clang' -j=32 -fix Reviewers: dblaikie Reviewed By: dblaikie Subscribers: klimek, cfe-commits Differential Revision: http://reviews.llvm.org/D8926 llvm-svn: 234678
2015-03-18Remove many superfluous SmallString::str() calls.Yaron Keren1-2/+2
Now that SmallString is a first-class citizen, most SmallString::str() calls are not required. This patch removes a whole bunch of them, yet there are lots more. There are two use cases where str() is really needed: 1) To use one of StringRef member functions which is not available in SmallString. 2) To convert to std::string, as StringRef implicitly converts while SmallString do not. We may wish to change this, but it may introduce ambiguity. llvm-svn: 232622
2015-01-14[cleanup] Re-sort *all* #include lines with llvm/utils/sort_includes.pyChandler Carruth1-1/+1
Sorry for the noise, I managed to miss a bunch of recent regressions of include orderings here. This should actually sort all the includes for Clang. Again, no functionality changed, this is just a mechanical cleanup that I try to run periodically to keep the #include lines as regular as possible across the project. llvm-svn: 225979
2014-11-19Update for LLVM API changeDavid Blaikie1-1/+1
llvm-svn: 222303
generated by cgit v1.1 at 2025-10-07 22:11:43 +0000