Age | Commit message (Collapse) | Author | Files | Lines |
|
This patch moves the virtual file system form clang to llvm so it can be
used by more projects.
Concretely the patch:
- Moves VirtualFileSystem.{h|cpp} from clang/Basic to llvm/Support.
- Moves the corresponding unit test from clang to llvm.
- Moves the vfs namespace from clang::vfs to llvm::vfs.
- Formats the lines affected by this change, mostly this is the result of
the added llvm namespace.
RFC on the mailing list:
http://lists.llvm.org/pipermail/llvm-dev/2018-October/126657.html
Differential revision: https://reviews.llvm.org/D52783
llvm-svn: 344140
|
|
llvm-svn: 343488
|
|
Summary: The convenience wrapper in STLExtras is available since rL342102.
Reviewers: rsmith, #clang, dblaikie
Reviewed By: rsmith, #clang
Subscribers: mgrang, arphaman, kadircet, cfe-commits
Differential Revision: https://reviews.llvm.org/D52576
llvm-svn: 343147
|
|
Summary:
Most callers I can find are using only `getName()`. Type is used by the
recursive iterator.
Now we don't have to call stat() on every listed file (on most platforms).
Exceptions are e.g. Solaris where readdir() doesn't include type information.
On those platforms we'll still stat() - see D51918.
The result is significantly faster (stat() can be slow).
My motivation: this may allow us to improve clang IO on large TUs with long
include search paths. Caching readdir() results may allow us to skip many stat()
and open() operations on nonexistent files.
Reviewers: bkramer
Subscribers: fedor.sergeev, cfe-commits
Differential Revision: https://reviews.llvm.org/D51921
llvm-svn: 342232
|
|
Summary:
Added support of creating a hardlink from one file to another file.
After a hardlink is added between two files, both file will have the same:
1. UniqueID (inode)
2. Size
3. Buffer
This will bring replay of compilation closer to the actual compilation. There are instances where clang checks for the UniqueID of the file/header to be loaded which leads to a different behavior during replay as all files have different UniqueIDs.
Patch by Utkarsh Saxena!
Reviewers: ilya-biryukov
Reviewed By: ilya-biryukov
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D51359
llvm-svn: 341366
|
|
llvm-svn: 339296
|
|
Entries with only a filename prevent us from building a file system tree and
cause the assertion
> Assertion failed: (NewParentE && "Parent entry must exist"), function uniqueOverlayTree, file clang/lib/Basic/VirtualFileSystem.cpp, line 1303.
Entries with a relative path are simply not discoverable during header search.
rdar://problem/28990865
Reviewers: bruno, benlangmuir
Reviewed By: bruno
Subscribers: dexonsmith, cfe-commits
Differential Revision: https://reviews.llvm.org/D49518
llvm-svn: 339164
|
|
requested name
Summary:
InMemoryFileSystem::status behaves differently than
RealFileSystem::status. The Name contained in the Status returned by
RealFileSystem::status will be the path as requested by the caller,
whereas InMemoryFileSystem::status returns the normalized path.
For example, when requested the status for "../src/first.h",
RealFileSystem returns a Status with "../src/first.h" as the Name.
InMemoryFileSystem returns "/absolute/path/to/src/first.h".
The reason for this change is that I want to make a unit test in the
clangd testsuite (where we use an InMemoryFileSystem) to reproduce a
bug I get with the clangd program (where a RealFileSystem is used).
This difference in behavior "hides" the bug in the unit test version.
An indirect impact of this change is that a -Wnonportable-include-path
warning is now emitted in test PCH/case-insensitive-include.c. This is
because the real path of the included file (with the wrong case) was not
available previously, whereas it is now.
Reviewers: malaperle, ilya-biryukov, bkramer
Reviewed By: ilya-biryukov
Subscribers: eric_niebler, malaperle, omtcyfz, hokein, bkramer, ilya-biryukov, ioeric, cfe-commits
Differential Revision: https://reviews.llvm.org/D48903
llvm-svn: 339063
|
|
Status with the requested name"
This broke clang/test/PCH/case-insensitive-include.c on Windows.
llvm-svn: 338084
|
|
requested name
Summary:
InMemoryFileSystem::status behaves differently than
RealFileSystem::status. The Name contained in the Status returned by
RealFileSystem::status will be the path as requested by the caller,
whereas InMemoryFileSystem::status returns the normalized path.
For example, when requested the status for "../src/first.h",
RealFileSystem returns a Status with "../src/first.h" as the Name.
InMemoryFileSystem returns "/absolute/path/to/src/first.h".
The reason for this change is that I want to make a unit test in the
clangd testsuite (where we use an InMemoryFileSystem) to reproduce a
bug I get with the clangd program (where a RealFileSystem is used).
This difference in behavior "hides" the bug in the unit test version.
Reviewers: malaperle, ilya-biryukov, bkramer
Subscribers: cfe-commits, ioeric, ilya-biryukov, bkramer, hokein, omtcyfz
Differential Revision: https://reviews.llvm.org/D48903
llvm-svn: 338057
|
|
the requested name"
This reverts commit r336807. This breaks users of
ClangTool::mapVirtualFile. Will try to investigate a fix. See also the
discussion on https://reviews.llvm.org/D48903
llvm-svn: 336831
|
|
requested name
Summary:
InMemoryFileSystem::status behaves differently than
RealFileSystem::status. The Name contained in the Status returned by
RealFileSystem::status will be the path as requested by the caller,
whereas InMemoryFileSystem::status returns the normalized path.
For example, when requested the status for "../src/first.h",
RealFileSystem returns a Status with "../src/first.h" as the Name.
InMemoryFileSystem returns "/absolute/path/to/src/first.h".
The reason for this change is that I want to make a unit test in the
clangd testsuite (where we use an InMemoryFileSystem) to reproduce a
bug I get with the clangd program (where a RealFileSystem is used).
This difference in behavior "hides" the bug in the unit test version.
In general, I guess it's good if InMemoryFileSystem works as much as
possible like RealFileSystem.
Doing so made the FileEntry::RealPathName value (assigned in
FileManager::getFile) wrong when using the InMemoryFileSystem. That's
because it assumes that vfs::File::getName will always return the real
path. I changed to to use FileSystem::getRealPath instead.
Subscribers: ilya-biryukov, ioeric, cfe-commits
Differential Revision: https://reviews.llvm.org/D48903
llvm-svn: 336807
|
|
The test uses unix paths and doesn't make sense to run on windows.
Fix bot failure caused by r333172:
http://lab.llvm.org:8011/builders/clang-x64-ninja-win7/builds/10799
llvm-svn: 333186
|
|
Reviewers: bkramer
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D47262
llvm-svn: 333172
|
|
Reviewers: bkramer
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D47060
llvm-svn: 332717
|
|
llvm-svn: 331177
|
|
Summary:
In https://reviews.llvm.org/D44960, file status check is executed every
time a real file system directory iterator is constructed or
incremented, and emits an error code. This change list fixes the errors
in VirtualFileSystem caused by https://reviews.llvm.org/D44960.
Patch by Yuke Liao (@liaoyuke).
Reviewers: vsk, pcc, zturner, liaoyuke
Reviewed By: vsk
Subscribers: mgrang, cfe-commits
Differential Revision: https://reviews.llvm.org/D45178
llvm-svn: 329223
|
|
r327219 added wrappers to std::sort which randomly shuffle the container before
sorting. This will help in uncovering non-determinism caused due to undefined
sorting order of objects having the same key.
To make use of that infrastructure we need to invoke llvm::sort instead of
std::sort.
llvm-svn: 328636
|
|
Summary:
In https://reviews.llvm.org/D39572 , I added support for specifying
`Type` when invoking `InMemoryFileSystem::addFile()`.
However, I didn't account for the fact that when `Type` is
`directory_file`, we need to construct an `InMemoryDirectory`, not an
`InMemoryFile`, or else clients cannot create files inside that
directory.
This diff fixes the bug and adds a test.
Test Plan: New test added. Ran test with:
% make -j12 check-clang-tools
Reviewers: bkramer, hokein
Reviewed By: bkramer
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D40140
llvm-svn: 318445
|
|
Summary:
This implements a FIXME in InMemoryFileSystem::addFile(), allowing
clients to specify User, Group, Type, and/or Perms when creating a
file in an in-memory filesystem.
New tests included. Ran tests with:
% ninja BasicTests && ./tools/clang/unittests/Basic/BasicTests
Fixes PR#35172 (https://bugs.llvm.org/show_bug.cgi?id=35172)
Reviewers: bkramer, hokein
Reviewed By: bkramer, hokein
Subscribers: alexfh
Differential Revision: https://reviews.llvm.org/D39572
llvm-svn: 317800
|
|
braces to avoid ambiguous 'else'. NFC.
llvm-svn: 305507
|
|
llvm-svn: 297759
|
|
Modified the tests to accept any iteration order, to run only on Unix, and added
additional error reporting to investigate SystemZ bot issue.
The VFS directory iterator and recursive directory iterator behave differently
from the LLVM counterparts. Once the VFS iterators hit a broken symlink they
immediately abort. The LLVM counterparts don't stat entries unless they have to
descend into the next directory, which allows to recover from this issue by
clearing the error code and skipping to the next entry.
This change adds similar behavior to the VFS iterators. There should be no
change in current behavior in the current CLANG source base, because all
clients have loop exit conditions that also check the error code.
This fixes rdar://problem/30934619.
Differential Revision: https://reviews.llvm.org/D30768
llvm-svn: 297693
|
|
Still broken on Windows and SystemZ bot ... sorry for the noise.
llvm-svn: 297533
|
|
llvm-svn: 297531
|
|
Modified the tests to accept any iteration order.
The VFS directory iterator and recursive directory iterator behave differently
from the LLVM counterparts. Once the VFS iterators hit a broken symlink they
immediately abort. The LLVM counterparts allow to recover from this issue by
clearing the error code and skipping to the next entry.
This change adds the same functionality to the VFS iterators. There should be
no change in current behavior in the current CLANG source base, because all
clients have loop exit conditions that also check the error code.
This fixes rdar://problem/30934619.
Differential Revision: https://reviews.llvm.org/D30768
llvm-svn: 297528
|
|
The tests are failing on one of the bots.
llvm-svn: 297517
|
|
The VFS directory iterator and recursive directory iterator behave differently
from the LLVM counterparts. Once the VFS iterators hit a broken symlink they
immediately abort. The LLVM counterparts allow to recover from this issue by
clearing the error code and skipping to the next entry.
This change adds the same functionality to the VFS iterators. There should be
no change in current behavior in the current CLANG source base, because all
clients have loop exit conditions that also check the error code.
This fixes rdar://problem/30934619.
Differential Revision: https://reviews.llvm.org/D30768
llvm-svn: 297510
|
|
Summary: NFCI
Reviewers: benlangmuir, zturner
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D25948
llvm-svn: 286356
|
|
Summary: Removed unused headers, replaced some headers with forward class declarations
Patch by: Eugene <claprix@yandex.ru>
Differential Revision: https://reviews.llvm.org/D20100
llvm-svn: 275882
|
|
Unlike sys::fs::recursive_directory_iterator,
vfs::recursive_directory_iterator does not implement the level() method,
which tells how deep in the directory tree the current iterator is. This
is needed in the vfs::recursive_directory_iterator so that future
improvements to the crash reproducer will be able to properly access
header for umbrellas when looking into the VFS.
rdar://problem/25880368
llvm-svn: 269520
|
|
Reapply r269100 and r269270, reverted due to
https://llvm.org/bugs/show_bug.cgi?id=27725. Isolate the testcase that
corresponds to the new feature side of this commit and skip it on
windows hosts until we find why it does not work on these platforms.
Original commit message:
The way we currently build the internal VFS overlay representation leads
to inefficient path search and might yield wrong answers when asked for
recursive or regular directory iteration.
Currently, when reading an YAML file, each YAML root entry is placed
inside a new root in the filesystem overlay. In the crash reproducer, a
simple "@import Foundation" currently maps to 43 roots, and when looking
up paths, we traverse a directory tree for each of these different
roots, until we find a match (or don't). This has two consequences:
- It's slow.
- Directory iteration gives incomplete results since it only return
results within one root - since contents of the same directory can be
declared inside different roots, the result isn't accurate.
This is in part fault of the way we currently write out the YAML file
when emitting the crash reproducer - we could generate only one root and
that would make it fast and correct again. However, we should not rely
on how the client writes the YAML, but provide a good internal
representation regardless.
Build a proper virtual directory tree out of the YAML representation,
allowing faster search and proper iteration. Besides the crash
reproducer, this potentially benefits other VFS clients.
llvm-svn: 269327
|
|
llvm-svn: 269326
|
|
accurate lookup"
Reverts r269270, buildbots still failing:
http://lab.llvm.org:8011/builders/clang-x64-ninja-win7/builds/12119
http://bb.pgr.jp/builders/ninja-clang-i686-msc19-R/builds/2847
llvm-svn: 269276
|
|
The way we currently build the internal VFS overlay representation leads
to inefficient path search and might yield wrong answers when asked for
recursive or regular directory iteration.
Currently, when reading an YAML file, each YAML root entry is placed
inside a new root in the filesystem overlay. In the crash reproducer, a
simple "@import Foundation" currently maps to 43 roots, and when looking
up paths, we traverse a directory tree for each of these different
roots, until we find a match (or don't). This has two consequences:
- It's slow.
- Directory iteration gives incomplete results since it only return
results within one root - since contents of the same directory can be
declared inside different roots, the result isn't accurate.
This is in part fault of the way we currently write out the YAML file
when emitting the crash reproducer - we could generate only one root and
that would make it fast and correct again. However, we should not rely
on how the client writes the YAML, but provide a good internal
representation regardless.
This patch builds a proper virtual directory tree out of the YAML
representation, allowing faster search and proper iteration. Besides the
crash reproducer, this potentially benefits other VFS clients.
llvm-svn: 269270
|
|
Do not rely on any specific order while comparing the results of
directory iteration.
llvm-svn: 269234
|
|
Bruno made a couple valiant attempts but the bot is still red.
This reverts r269100 (primary commit), r269108 (fix attempt), r269133
(fix attempt).
llvm-svn: 269160
|
|
http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/builds/5110
Follow up from r269100.
llvm-svn: 269133
|
|
http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/builds/5103
Follow up from r269100.
llvm-svn: 269108
|
|
The way we currently build the internal VFS overlay representation leads
to inefficient path search and might yield wrong answers when asked for
recursive or regular directory iteration.
Currently, when reading an YAML file, each YAML root entry is placed
inside a new root in the filesystem overlay. In the crash reproducer, a
simple "@import Foundation" currently maps to 43 roots, and when looking
up paths, we traverse a directory tree for each of these different
roots, until we find a match (or don't). This has two consequences:
- It's slow.
- Directory iteration gives incomplete results since it only return
results within one root - since contents of the same directory can be
declared inside different roots, the result isn't accurate.
This is in part fault of the way we currently write out the YAML file
when emitting the crash reproducer - we could generate only one root and
that would make it fast and correct again. However, we should not rely
on how the client writes the YAML, but provide a good internal
representation regardless.
This patch builds a proper virtual directory tree out of the YAML
representation, allowing faster search and proper iteration. Besides the
crash reproducer, this potentially benefits other VFS clients.
llvm-svn: 269100
|
|
This reapplies r261552 and r263748. Fixed testcase to reapply.
The VFS overlay mapping between virtual paths and real paths is done through
the 'external-contents' entries in YAML files, which contains hardcoded paths
to the real files.
When a module compilation crashes, headers are dumped into <name>.cache/vfs
directory and are mapped via the <name>.cache/vfs/vfs.yaml. The script
generated for reproduction uses -ivfsoverlay pointing to file to gather the
mapping between virtual paths and files inside <name>.cache/vfs. Currently, we
are only capable of reproducing such crashes in the same machine as they
happen, because of the hardcoded paths in 'external-contents'.
To be able to reproduce a crash in another machine, this patch introduces a new
option in the VFS yaml file called 'overlay-relative'. When it's equal to
'true' it means that the provided path to the YAML file through the
-ivfsoverlay option should also be used to prefix the final path for every
'external-contents'.
Example, given the invocation snippet "... -ivfsoverlay
<name>.cache/vfs/vfs.yaml" and the following entry in the yaml file:
"overlay-relative": "true",
"roots": [
...
"type": "directory",
"name": "/usr/include",
"contents": [
{
"type": "file",
"name": "stdio.h",
"external-contents": "/usr/include/stdio.h"
},
...
Here, a file manager request for virtual "/usr/include/stdio.h", that will map
into real path "/<absolute_path_to>/<name>.cache/vfs/usr/include/stdio.h.
This is a useful feature for debugging module crashes in machines other than
the one where the error happened.
Differential Revision: http://reviews.llvm.org/D17457
rdar://problem/24499339
llvm-svn: 263893
|
|
Tests failing on
http://bb.pgr.jp/builders/cmake-clang-x86_64-linux/builds/46102
This reverts commit a1683cd6c9e07359c09f86e98a4db6b4e1bc51fc.
llvm-svn: 263750
|
|
This reapplies r261552.
The VFS overlay mapping between virtual paths and real paths is done through
the 'external-contents' entries in YAML files, which contains hardcoded paths
to the real files.
When a module compilation crashes, headers are dumped into <name>.cache/vfs
directory and are mapped via the <name>.cache/vfs/vfs.yaml. The script
generated for reproduction uses -ivfsoverlay pointing to file to gather the
mapping between virtual paths and files inside <name>.cache/vfs. Currently, we
are only capable of reproducing such crashes in the same machine as they
happen, because of the hardcoded paths in 'external-contents'.
To be able to reproduce a crash in another machine, this patch introduces a new
option in the VFS yaml file called 'overlay-relative'. When it's equal to
'true' it means that the provided path to the YAML file through the
-ivfsoverlay option should also be used to prefix the final path for every
'external-contents'.
Example, given the invocation snippet "... -ivfsoverlay
<name>.cache/vfs/vfs.yaml" and the following entry in the yaml file:
"overlay-relative": "true",
"roots": [
...
"type": "directory",
"name": "/usr/include",
"contents": [
{
"type": "file",
"name": "stdio.h",
"external-contents": "/usr/include/stdio.h"
},
...
Here, a file manager request for virtual "/usr/include/stdio.h", that will map
into real path "/<absolute_path_to>/<name>.cache/vfs/usr/include/stdio.h.
This is a useful feature for debugging module crashes in machines other than
the one where the error happened.
Differential Revision: http://reviews.llvm.org/D17457
rdar://problem/24499339
llvm-svn: 263748
|
|
call to getVFSFromYAML in unittests"
This reverts commit r261552 and r261556 because of failing unittests on
windows:
Failing Tests (4):
Clang-Unit :: Basic/BasicTests.exe/VFSFromYAMLTest.CaseInsensitive
Clang-Unit :: Basic/BasicTests.exe/VFSFromYAMLTest.DirectoryIteration
Clang-Unit :: Basic/BasicTests.exe/VFSFromYAMLTest.MappedFiles
Clang-Unit :: Basic/BasicTests.exe/VFSFromYAMLTest.UseExternalName
llvm-svn: 261613
|
|
Follow up from r261552
llvm-svn: 261556
|
|
Differential revision: http://reviews.llvm.org/D16567
llvm-svn: 258836
|
|
No functional change intended.
llvm-svn: 257287
|
|
llvm-svn: 257286
|
|
FixedCompilationDatabase sets the working dir to "." by default. For
chdir(".") this is a noop but this lead to InMemoryFileSystem to create
bogus paths. Fixes PR25327.
llvm-svn: 257260
|
|
Make RedirectedFileSystem::openFilForRead(path)->status() the same as
RedirectedFileSystem::status(path). Previously we would just get the
status of the underlying real file, which would not have the IsVFSMapped
bit set.
This fixes rebuilding a module that has an include that is relative to
the includer where we will lookup the real path of that file before we
lookup the VFS location.
rdar://problem/23640339
llvm-svn: 255312
|