| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Add MappedFileRegionArena which can be served as a file system backed
persistent memory allocator. The allocator works like a
BumpPtrAllocator,
and is designed to be thread safe and process safe.
The implementation relies on the POSIX compliance of file system and
doesn't work on all file systems. If the file system supports lazy tail
(doesn't allocate disk space if the tail of the large file is not used),
user has more flexibility to declare a larger capacity.
The allocator works by using a atomically updated bump ptr at a location
that can be customized by the user. The atomic pointer points to the
next available space to allocate, and the allocator will resize/truncate
to current usage once all clients closed the allocator.
Windows implementation contributed by: @hjyamauchi
|
|
Add parameter to file lock API to allow exclusive file lock. Both Unix
and Windows support lock the file exclusively for write for one process
and LLVM OnDiskCAS uses exclusive file lock to coordinate CAS creation.
|
|
|
|
NumRead is already of ssize_t.
|
|
This patch removes the C++
disablezOSAutoConversion,enablezOSAutoConversion declarations and also
updates Path.inc to use the common function.
|
|
|
|
This PR is part of a series to upstream managarm support, as laid out in
the
[RFC](https://discourse.llvm.org/t/rfc-new-proposed-managarm-support-for-llvm-and-clang-87845/85884/1).
This PR is a follow-up to #87845.
The intention here is to add the managarm target to the LLVM support
lib, in preparation for a follow-up that would add the missing clang
driver bits.
|
|
Closes #124652.
This header was introduced in
https://github.com/llvm/llvm-project/commit/536736995bf5d073853c7e884968c9847b4ae64d,
but it appears that including only `fnctl.h` should be enough.
Hopefully, this patch will not cause build issues on other Unix
platforms.
|
|
It is sufficient to just use `HAVE_DLOPEN`.
|
|
These are unneeded even on AIX, PURE_WINDOWS, and ZOS (per #104706)
* HAVE_ERRNO_H: introduced by 1a93330ffa2ae2aa0b49461f05e6f0d51e8443f8 (2009) but unneeded.
The guarded ABI is unconditionally used by lldb.
* HAVE_FCNTL_H
* HAVE_FENV_H
* HAVE_SYS_STAT_H
Pull Request: https://github.com/llvm/llvm-project/pull/123087
|
|
f4719c4d2cda ("Add support for GNU Hurd in Path.inc and other places")
made llvm use an internal __f_type name for the f_type field (which it
is not supposed to since accessing double-underscore names is explicitly
not supported by standards). In glibc 2.39 this field was renamed to
f_type so application can now access the field as the standard says.
|
|
generic (#107399)
This patch renames the functions in AutoConvert.h/cpp to have a less
generic name because they are z/OS specific.
|
|
The removal of StringRef::equals in
3fa409f2318ef790cc44836afe9a72830715ad84 broke the
[Solaris/sparcv9](https://lab.llvm.org/buildbot/#/builders/13/builds/724)
and
[Solaris/amd64](https://lab.llvm.org/staging/#/builders/94/builds/5176)
buildbots:
```
In file included from /vol/llvm/src/llvm-project/git/llvm/lib/Support/Path.cpp:1200:
/vol/llvm/src/llvm-project/git/llvm/lib/Support/Unix/Path.inc:519:18: error: no member named 'equals' in 'llvm::StringRef'
519 | return !fstype.equals("nfs");
| ~~~~~~ ^
```
Fixed by switching to `operator!=` instead.
Tested on sparcv9-sun-solaris2.11 and amd64-pc-solaris2.11.
|
|
The test `llvm/unittests/Support/CommandLineTest.cpp` that handles
errors in expansion of response files was previously disabled for AIX.
Originally the code was dependent on `read` returning `EISDIR` which
occurs on platforms such as Linux. However, other platforms such as AIX
allow use of `read` on file descriptors for directories. This change
updates `readNativeFile` to produce `EISDIR` on AIX and z/OS when used
on a directory (instead of relying on the call to `read` to do so).
---------
Co-authored-by: Alison Zhang <alisonzhang@ibm.com>
Co-authored-by: James Henderson <46713263+jh7370@users.noreply.github.com>
|
|
LLVM is inconsistent about how it converts `errno` to `std::error_code`.
This can cause problems because values outside of `std::errc` compare
differently if one is system and one is generic on POSIX systems.
This is even more of a problem on Windows where use of the system
category is just wrong, as that is for Windows errors, which have a
completely different mapping than POSIX/generic errors. This patch fixes
one instance of this mistake in `JSONTransport.cpp`.
This patch adds `errnoAsErrorCode()` which makes it so people do not
need to think about this issue in the future. It also cleans up a lot of
usage of `errno` in LLVM and Clang.
|
|
This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,ends}_with in C++20.
I'm planning to deprecate and eventually remove
StringRef::{starts,ends}with.
|
|
This uses argv[0] on Haiku.
|
|
This uses just argv[0] on OpenBSD. OpenBSD does not support the /proc filesystem.
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D159168
|
|
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D158806
|
|
Differential Revision: https://reviews.llvm.org/D137800
|
|
Apply clang-format on llvm/lib/Support/Windows/ and llvm/lib/Support/Unix/ since .inc files in these folders aren't picked up by default. Eventually we need to add this extension in the monorepo .clang-format file.
Differential Revision: https://reviews.llvm.org/D138714
|
|
OpenGroup specification doesn't require getpwuid and getpwnam
to be thread-safe. And musl libc has a not thread-safe implementation.
When building clang with musl, this can make clang-scan-deps crash.
Reviewed By: pirama
Differential Revision: https://reviews.llvm.org/D137864
|
|
This reverts commit 142f51fc2f448845f6a32e767ffaa2b665eea11f.
This shouldn't be committed, it got committed accidentally.
|
|
Add mapped_file_region::sync(), equivalent to POSIX msync,
synchronizing written content to disk without unmapping the region.
Asserts if the mode is not mapped_file_region::readwrite.
Note that I don't have access to a Windows machine, so I can't
easily run those unit tests.
Change by dexonsmith
Differential Revision: https://reviews.llvm.org/D95494
|
|
We were dereferencing an empty Optional if IgnoreErrors was true and the
stat failed.
rdar://60887887
Differential Revision: https://reviews.llvm.org/D131791
|
|
This patch adds an llvm-driver multicall tool that can combine multiple
LLVM-based tools. The build infrastructure is enabled for a tool by
adding the GENERATE_DRIVER option to the add_llvm_executable CMake
call, and changing the tool's main function to a canonicalized
tool_name_main format (i.e. llvm_ar_main, clang_main, etc...).
As currently implemented llvm-driver contains dsymutil, llvm-ar,
llvm-cxxfilt, llvm-objcopy, and clang (if clang is included in the
build).
llvm-driver can be enabled from builds by setting
LLVM_TOOL_LLVM_DRIVER_BUILD=On.
There are several limitations in the current implementation, which can
be addressed in subsequent patches:
(1) the multicall binary cannot currently properly handle
multi-dispatch tools. This means symlinking llvm-ranlib to llvm-driver
will not properly result in llvm-ar's main being called.
(2) the multicall binary cannot be comprised of tools containing
conflicting cl::opt options as the global cl::opt option list cannot
contain duplicates.
These limitations can be addressed in subsequent patches.
Differential revision: https://reviews.llvm.org/D109977
|
|
In contrast to Linux it does not provide entries which can be readlinked
-- these are just regular files, not giving the expected outcome. That's
on top of procfs not being mounted by default to begin with.
This is probably the case on other BSDs as well, so I expect there will
be more ifdefs added down the road.
Reviewed By: emaste, dim
Differential Revision: https://reviews.llvm.org/D122545
|
|
This is a followup to D119695 using the suggestion by joerg. Rather
than manually declaring madvise() on __sun__, this uses
posix_madvise() if available, which does get declared properly on
Illumos.
Differential Revision: https://reviews.llvm.org/D119856
|
|
D116366 added a call to madvise() in Path.inc. Unfortunately,
Illumos does not declare this function if _XOPEN_SOURCE is defined
(which it is by default) and we need to provide the declaration
manually. This is the same workaround used in sanitizers:
https://github.com/llvm/llvm-project/blob/ee423d93ead39e94c2970b3cc7ef6e6faa75d10b/compiler-rt/lib/sanitizer_common/sanitizer_solaris.cpp#L77-L85
Differential Revision: https://reviews.llvm.org/D119695
|
|
Replace a `reserve()`/`set_size()` pair with `resize_for_overwrite()`
and `truncate()`. The out parameter also needs a `clear()` call on the
error path.
Differential Revision: https://reviews.llvm.org/D115389
|
|
mapped_file_region::dontNeedImpl implementation
mapped_file_region::dontNeedImpl added in D116366 calls madvise, which
causes problems for z/OS and AIX.
For z/OS, we don't have either madvise, so treat this as a no-op, same
as Windows does.
For AIX, it doesn't have any effect, doesn't have a standardized
signature, and it needs certain feature test macros (i.e. _ALL_SOURCE)
we don't set by default for LLVM on AIX, so just make it a no-op too.
Differential Revision: https://reviews.llvm.org/D116603
|
|
On *NIX systems, this API calls madvise(MADV_DONTNEED) on read-only file mappings.
It should not be used on a writable buffer.
The API is used to implement ld.lld LTO memory saving trick (D116367).
Note: on read-only file mappings, Linux's MADV_DONTNEED semantics match POSIX
POSIX_MADV_DONTNEED and BSD systems' MADV_DONTNEED.
On Windows, VirtualAllocEx MEM_COMMIT/MEM_RESET have similar semantics
but are unfortunately not drop-in replacements. dontNeedIfMmap is currently a no-op.
Reviewed By: aganea
Differential Revision: https://reviews.llvm.org/D116366
|
|
Identified with modernize-use-nullptr.
|
|
This reverts 3816c53f040cc6aa06425978dd504b0bd5b7899c and removes follow-up
fixups.
The original intention was to show error earlier (posix_fallocate time) than
later for ld.lld but it appears to cause some problems which make it not free.
* FreeBSD ZFS: EINVAL, not too bad.
* FreeBSD UFS: according to khng "devastatingly slow on freebsd because UFS on freebsd does not have preallocation support like illumos. It zero-fills."
* NetBSD: maybe EOPNOTSUPP
* Linux tmpfs: unless tmpfs is set up to use huge pages (requires CONFIG_TRANSPARENT_HUGE_PAGECACHE=y), I can consistently demonstrate ~300ms delay for a 1.4GiB output.
* Linux ext4: I don't measure any benefit, either backed by a hard disk or by a file in tmpfs.
* The current code organization of `defined(HAVE_POSIX_FALLOCATE)` costs us a macro dispatch for AIX.
I think we should just remove it. I think if posix_fallocate ever finds demonstrable benefit,
it is likely Linux specific and will not need HAVE_POSIX_FALLOCATE, and possibly opt-in by some specific programs.
In a filesystem with CoW and compression, the ENOSPC benefit may be lost as well.
Reviewed By: khng300
Differential Revision: https://reviews.llvm.org/D115957
|
|
fs::copy_file() on Darwin has a nice optimization to clone the file when
possible. Change the implementation to use clonefile() directly, instead
of the higher-level copyfile(). The latter does the wrong thing for
symlinks, which requires calling `stat` first...
With that out of the way, optimistically call clonefile() all the time,
and then for any error that's recoverable try again with copyfile()
(without the COPYFILE_CLONE flag, as before).
Differential Revision: https://reviews.llvm.org/D112250
|
|
Many `flang` tests currently `FAIL` on Solaris because the module files
aren't found. I could trace this to `sys::fs::getMainExecutable` not being
implemented.
This patch does this and fixes all affected `flang` tests.
Tested on `amd64-pc-solaris2.11`.
Differential Revision: https://reviews.llvm.org/D109374
|
|
On FreeBSD, absolute paths are passed unmodified in AT_EXECPATH, but
relative paths are resolved to absolute paths, and any symlinks will be
followed in the process. This means that the resource dir calculation
will be wrong if Clang is invoked as an absolute path to a symlink, and
this currently causes clang/test/Driver/rocm-detect.hip to fail on
FreeBSD. Thus, make sure to call realpath on the result, just like is
done on macOS.
Whilst here, clean up the old fallback auxargs loop to use the actual
type for auxargs rather than using lots of hacky casts that rely on
addresses and pointers being the same (which is not the case on CHERI,
and thus Arm's prototype Morello, although for little-endian systems it
happens to work still as the word-sized integer will be padded to a full
pointer, and it's someone academic given dereferencing past the end of
environ will give a bounds fault, but CheriBSD is new enough that the
elf_aux_info path will be used). This also makes the code easier to
follow, and removes the confusing double-increment of p.
Reviewed By: dim, arichardson
Differential Revision: https://reviews.llvm.org/D103346
|
|
This patch adds the basic functions needed for controlling auto conversion on z/OS.
Auto conversion is enabled on untagged input file to ASCII by making the assumption that all untagged files are EBCDIC encoded. Output files are auto converted to EBCDIC IBM-1047.
This change also enables conversion for stdin/stdout/stderr.
For more information on how fcntl controls codepage https://www.ibm.com/docs/en/zos/2.4.0?topic=descriptions-fcntl-bpx1fct-bpx4fct-control-open-file-descriptors
Reviewed By: anirudhp
Differential Revision: https://reviews.llvm.org/D100483
|
|
Update llvm::sys::fs::mapped_file_region to have a move constructor and
a move assignment operator, allowing it to be used as an Optional. Also,
update FileOutputBuffer's OnDiskBuffer to take advantage of this,
avoiding an extra allocation from the unique_ptr.
A nice follow-up would be to make the mapped_file_region constructor
private and replace its use with a factory function, such as
mapped_file_region::create(), that returns an Expected (or ErrorOr). I
don't plan on doing that immediately, but I might swing back later.
No functionality change, besides the saved allocation in OnDiskBuffer.
Differential Revision: https://reviews.llvm.org/D100159
|
|
|
|
This allows mapping larger files, delaying OOM failures until too many
pages of them are accessed. This is makes the behavior of the
mapped_file_region in this regard consistent between its "Unix" and
"Windows" implementations.
Guard the code witih #if defined(MAP_NORESERVE), consistent with other
uses of MAP_NORESERVE in llvm-project, because some FreeBSD versions do
not provide this flag.
Reviewed By: clayborg
Differential Revision: https://reviews.llvm.org/D96626
|
|
As of binutils 2.36, GNU strip calls chown(2) for "sudo strip foo" and
"sudo strip foo -o foo", but no "sudo strip foo -o bar" or "sudo strip
foo -o ./foo". In other words, while "sudo strip foo -o bar" creates a
new file bar with root access, "sudo strip foo" will keep the owner and
group of foo unchanged. Currently llvm-objcopy and llvm-strip behave
differently, always changing the owner and gropu to root. The
discrepancy prevents Chrome OS from migrating to llvm-objcopy and
llvm-strip as they change file ownership and cause intended users/groups
to lose access when invoked by sudo with the following sequence
(recommended in man page of GNU strip).
1.<Link the executable as normal.>
1.<Copy "foo" to "foo.full">
1.<Run "strip --strip-debug foo">
1.<Run "objcopy --add-gnu-debuglink=foo.full foo">
This patch makes llvm-objcopy and llvm-strip follow GNU's behavior.
Link: crbug.com/1108880
|
|
Identified with readability-redundant-control-flow.
|
|
When LLDB Python bindings are used and stack backtraces are enabled
for logging, getMainExecutable() is called with argv0 being null.
This caused the fallback function getprogpath() (used on FreeBSD, NetBSD
and Linux) to segfault. Make it handle null executable name gracefully.
Differential Revision: https://reviews.llvm.org/D91012
|
|
On macOS, the read and pread syscalls return EINVAL when the number of
bytes to read exceeds INT32_MAX:
https://github.com/apple/darwin-xnu/blob/a449c6a3b8014d9406c2ddbdc81795da24aa7443/bsd/kern/sys_generic.c#L355
rdar://68751407
Differential revision: https://reviews.llvm.org/D90201
|
|
loading modules.""
This reverts commit 4000c9ee18ecebe3ff0f197af8c1fb434ad986e5.
Test "LLVM :: Other/statistic.ll" is failing on Windows.
|
|
modules."
Measure amount of high-level or fixed-cost operations performed during
building/loading modules and during header search. High-level operations
like building a module or processing a .pcm file are motivated by
previous issues where clang was re-building modules or re-reading .pcm
files unnecessarily. Fixed-cost operations like `stat` calls are tracked
because clang cannot change how long each operation takes but it can
perform fewer of such operations to improve the compile time.
Also tracking such stats over time can help us detect compile-time
regressions. Added stats are more stable than the actual measured
compilation time, so expect the detected regressions to be less noisy.
On relanding drop stats in MemoryBuffer.cpp as their value is pretty low
but affects a lot of clients and many of those aren't interested in
modules and header search.
rdar://problem/55715134
Reviewed By: aprantl, bruno
Differential Revision: https://reviews.llvm.org/D86895
|
|
modules."
This reverts commit c4bacc3c9b333bb7032fb96f41d6f5b851623132.
Test "LLVM :: ThinLTO/X86/funcimport-stats.ll" is failing. Reverting now
and will recommit after making the test not fail with the added stats.
|
|
Measure amount of high-level or fixed-cost operations performed during
building/loading modules and during header search. High-level operations
like building a module or processing a .pcm file are motivated by
previous issues where clang was re-building modules or re-reading .pcm
files unnecessarily. Fixed-cost operations like `stat` calls are tracked
because clang cannot change how long each operation takes but it can
perform fewer of such operations to improve the compile time.
Also tracking such stats over time can help us detect compile-time
regressions. Added stats are more stable than the actual measured
compilation time, so expect the detected regressions to be less noisy.
rdar://problem/55715134
Reviewed By: aprantl, bruno
Differential Revision: https://reviews.llvm.org/D86895
|
|
The function setLastAccessAndModificationTime() uses function
futimens() or futimes() by default. Both functions are not
available in z/OS, therefore functionality is implemented using
__fchattr() on z/OS.
Reviews by: abhina.sreeskantharajan
Differential Revision: https://reviews.llvm.org/D83945
|
