Fixes: #94634

Call stacks are a huge portion of the MemProf profile, taking up 70+%
of the profile file size.

This patch implements a radix tree to compress call stacks, which are
known to have long common prefixes.  Specifically,
CallStackRadixTreeBuilder, introduced in this patch, takes call stacks
in the MemProf profile, sorts them in the dictionary order to maximize
the common prefix between adjacent call stacks, and then encodes a
radix tree into a single array that is ready for serialization.

The resulting radix array is essentially a concatenation of call stack
arrays, each encoded with its length followed by the payload, except
that these arrays contain "instructions" like "skip 7 elements
forward" to borrow common prefixes from other call stacks.

This patch does not integrate with the MemProf
serialization/deserialization infrastructure yet.  Once integrated,
the radix tree is expected to roughly halve the file size of the
MemProf profile.

This patch fixes a build issue following e57308b0

 when enabling
module build.

With that change, we failed to build the LLVM_IR module since
GEPNoWrapFlags wasn't defined prior to using it.

This patch addressed that issue by including the missing header in
`llvm/IR/IRBuilderFolder.h` which uses the `GEPNoWrapFlags` type.
This should ensure that we can always build the `LLVM_IR` module.

Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>

Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>

This function is called during very early startup and which can result
in a crash on FreeBSD. The sigaction() function in libc is indirected
via a table so that it can be interposed by the threading library
rather than calling the syscall directly. In the crash I was observing
this table had not yet been relocated, so we ended up jumping to an
invalid address. To avoid this problem we can call __sys_sigaction,
which calls the syscall directly and in FreeBSD 15 is part of libsys
rather than libc, so does not depend on libc being fully initialized.

…ty with high-entropy ASLR

With high-entropy ASLR (e.g., 32-bits == 16TB), the allocator base of
0x700000000000 (112TB) may collide with the placement of the libraries
(e.g., on Linux, the mmap base could be 128TB - 16TB == 112TB). This
results in a segfault in the test case.

This patch moves the allocator base below the PIE program segment,
inspired by fb77ca05. As per that patch:
1) we are leaving the old behavior for Apple 2) since ASLR cannot be set
above 32-bits for x86-64 Linux, we expect this new layout to be durable.

Note that this is only changing a test case, not the behavior of
sanitizers. Sanitizers have their own settings for initializing the
allocator base.

Reproducer:
1. ninja check-sanitizer # Just to build the test binary needed below;
no need to actually run the tests here
2. sudo sysctl vm.mmap_rnd_bits=32 # Increase ASLR entropy
3. for f in `seq 1 10000`; do echo $f;
GTEST_FILTER=*SizeClassAllocator64Dense
./projects/compiler-rt/lib/sanitizer_common/tests/Sanitizer-x86_64-Test
> /tmp/x; if [ $? -ne 0 ]; then cat /tmp/x; fi; done

These CHECKs are all checking indices, which must be strictly smaller
than the size (otherwise they would go out of bounds).

- addressed
https://github.com/llvm/llvm-project/pull/94317#issuecomment-2153103129


- added conditional in cmake file for exit_handler object library

Co-authored-by: Aaryan Shukla <aaryanshukla@google.com>

This patch replaces llvm::SmallVector<Frame> with std::vector<Frame>.

llvm::SmallVector<Frame> sets aside one inline element.  Meanwhile,
when I sort all call stacks by their lengths, the length at the first
percentile is already 2.  That is, 99 percent of call stacks do not
take advantage of the inline element.

Using std::vector<Frame> reduces the cycle and instruction counts by
11% and 22%, respectively, with "llvm-profdata show" modified to
deserialize all MemProfRecords.

Fixed for more accurate searches of the flag `-Wsystem-headers-in-module=`.

This patch removes swapToHostOrder in favor of
llvm::support::endian::readNext as swapToHostOrder is too thin a
wrapper around readNext.

Note that there are two variants of readNext:

- readNext<type, endian, align>(ptr)
- readNext<type, align>(ptr, endian)

swapToHostOrder uses the former, but this patch switches to the latter.

While we are at it, this patch teaches readNext to default to
unaligned just as I did in:

  commit 568368a4
  Author: Kazu Hirata <kazu@google.com>
  Date:   Mon Apr 15 19:05:30 2024 -0700

getVNInfoFromReg is expected to return a nullptr if-and-only-if the
operand is undef.  (This was asserted for.)  Reverse the order of the
checks to simplify an upcoming set of patches.

Summary:
The old COV3 implementation of HSA used to omit the implicit arguments
from the kernel argument size. For COV4 and COV5 this is no longer the
case so we can simply use the size reported from the symbol information.

See
https://github.com/ROCm/ROCR-Runtime/issues/117#issuecomment-812758161

The summary already includes other size information, e.g. total debug
info size in bytes. The only other way I can get this information is by
dumping all statistics which can be quite large. Adding it to the
summary seems fair.

Relaxes restriction that certain public utility functions only apply
to the builtin ModuleOp.

Remove `REQUIRES: shell` from some tests that seem fine without it.
Tested on Windows and with LIT_USE_INTERNAL_SHELL=1 on Linux.

Summary:
We don't have the abs function to link against, just use the builtin.

This PR removes the `target-aarch64` requirement on the crashlog tests
to exercice them on Intel bots and make image loading single-threaded
temporarily while implementing a fix for a deadlock issue when loading
the images in parallel.

Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>

https://reviews.llvm.org/D85867 changed the way we assign file offsets
(alloc sections first, then non-alloc sections).

It also removed a non-alloc special case from `findOrphanPos`.
Looking at the memory-nonalloc-no-warn.test change, which would be
needed by #93761, it makes sense to restore the previous behavior: when
placing non-alloc orphan sections, keep these sections at the end so
that the section index order matches the file offset order.

This change is cosmetic. In sections-nonalloc.s, GNU ld places the
orphan `other3` in the middle and the orphan .symtab/.shstrtab/.strtab
at the end.

Pull Request: https://github.com/llvm/llvm-project/pull/94519

Changing the type of Frame::SymbolName from std::optional<std::string>
to std::unique<std::string> reduces sizeof(Frame) from 64 to 32.

The smaller type reduces the cycle and instruction counts by 23% and
4.4%, respectively, with "llvm-profdata show" modified to deserialize
all MemProfRecords in a MemProf V2 profile.  The peak memory usage is
cut down nearly by half.

This code uses namespaces `llvm` and `llvm::json`. However, we have both
`llvm::Value` and `llvm::json::Value`. Whenever any of the headers
declare or include `llvm::Value`, the lookup becomes ambiguous.

Fixing this by qualifying the `Value` type.

This adds:
- A ctor accepting a start and end iterator
- A ctor accepting a count and const T&
- size()
- subscript operators
- begin() and end() iterators

Derived from #92480. This PR introduces reduction semantics into loops
for DO CONCURRENT REDUCE. The `fir.do_loop` operation now invisibly has
the `operandSegmentsizes` attribute and takes variable-length reduction
operands with their operations given as `fir.reduce_attr`. For the sake
of compatibility, `fir.do_loop`'s builder has additional arguments at
the end. The `iter_args` operand should be placed in front of the
declaration of result types, so the new operand for reduction variables
(`reduce`) is put in the middle of arguments.

The ContextIds set on the ContextNode struct is not technically needed
as we can compute it from either the callee or caller edge context ids.
Remove it and add a helper to recompute from the edges on demand. Also
add helpers to compute the node allocation type and whether the context
ids are empty from the edges without needing to first compute the node's
context id set, to minimize the runtime cost increase.

This yielded a 20% reduction in peak memory for a large thin link, for
about a 2% time increase (which is more than offset by some other recent
time efficiency improvements).

We named the intrinsics by replacing "." by "_" in the instruction
conventionally, so the `vcpopv_v` where the corresponding instruction is
`vcpop.v` should be named `vcpop_v`.

The fold will replace 2 uses of `Y` we should also do fold if `Y` has
2 uses (not only oneuse).

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D159062

We only get a "reached end of constexpr function" diagnostic
otherwise.

This patch implements the following change to enable zos for death test
support. https://github.com/google/googletest/pull/4527

- added at_quick_exit function 
- used helper file exit_handler which reuses code from atexit
- atexit now calls helper functions from exit_handler
- test cases and dependencies are added

---------

Co-authored-by: Aaryan Shukla <aaryanshukla@google.com>

Fixed crash caused by call to getCookedLiteral on
template user defined literal. Fix base on assert in getCookedLiteral
method.

Closes #94454