We currently use ad-hoc spin waiting to synchronize thread creation
and thread start both ways. But spinning tend to degrade ungracefully
under high contention (lots of threads are created at the same time).
Use semaphores for synchronization instead.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D107337

This patch implements P2092

Simple requirements in requirement body shall not start with requires.
A warning was already in place so we just turn this warning into an error.

In addition, we add tests to make sure typename is optional in
requirement-parameter-list as per the same paper.

As detailed on https://pvs-studio.com/en/blog/posts/cpp/0771/ and raised on D62583, the SecNo++ increment is not guaranteed to occur before the second use of SecNo in the same addSection() call.

This patch pulls out the increment (just for clarity) and replaces the second use of SecNo with a constant zero value (we're using stable_sort so the value isn't critical).

Differential Revision: https://reviews.llvm.org/D107273

Next step is to generate an archive with all implementations and a header listing them all.

Differential Revision: https://reviews.llvm.org/D107336

Replace more raw types with Tid/StackID typedefs.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D107335

Previously we would show an error, but keep the member, and also the
CXXRrecordDecl, valid. This could lead to crashes when attempting to
access the record layout or size.

Differential Revision: https://reviews.llvm.org/D105478

This patch adds the critical construct to the OpenMP dialect. The
implementation models the definition in 2.17.1 of the OpenMP 5 standard.
A name and hint can be specified. The name is a global entity or has
external linkage, it is modelled as a FlatSymbolRefAttr. Hint is
modelled as an integer enum attribute.
Also lowering to LLVM IR using the OpenMP IRBuilder.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D107135

This patch extends the optimization of VID-sequence BUILD_VECTORs
introduced in D104921 to include simple fractional steps composed of a
separated integer numerator and denominator.

A notable limitation in this sequence detection is that only sequences
with steps N/1 or 1/D are found, meaning that the step between elements
and the frequency with which it changes is consistent across the whole
sequence. Fractional steps such as 2/3 won't be matched as those would
involve more complex tracking of state or some level of backtracking.

As is stands, however, this patch is sufficient to match common
interleave-type shuffle indices, for example matching `<0,0,1,1>` (or
commonly `<0,u,1,u>` or `<u,0,u,1>`) to an index sequence divided by 2.

While the optimization is relatively `undef`-tolerant, due to greedy
pattern-matching there even are some simple patterns which confuse the
sequence detection into identifying either a suboptimal sequence or no
sequence at all.

Currently only fractional-step sequences identified as having a
power-of-two denominator are actually lowered to RVV instructions. This
is to avoid introducing divisions into the generated code.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D106533

Differential Revision: https://reviews.llvm.org/D107332

Add a comment when there is a shifted value,
    add x9, x0, #291, lsl #12 ; =1191936
but not when the immediate value is unshifted,
    subs x9, x0, #256 ; =256
when the comment adds nothing additional to the reader.

Differential Revision: https://reviews.llvm.org/D107196

Store both interfaceID and objectID as key for interface registration callback.
Otherwise the implementation allows to register only one external model per one object in the single dialect.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D107274

The patch https://reviews.llvm.org/D105964 (https://github.com/llvm/llvm-project/commit/58494c856a15f5b0e886c7baf5d505ac6c05dfe5)
updated detection of function declaration names. It had the unfortunate
consequence that it started breaking between `function` and the function
name in some cases in JavaScript code.

This patch addresses this.

Reviewed By: MyDeveloperDay, owenpan

Differential Revision: https://reviews.llvm.org/D107267

Currently unaligned access functions are defined in tsan_interface.cpp
and do a real call to MemoryAccess. This means we have a real call
and no read/write constant propagation.

Unaligned memory access can be quite hot for some programs
(observed on some compression algorithms with ~90% of unaligned accesses).

Move them to tsan_interface_inl.h to avoid the additional call
and enable constant propagation.
Also reorder the actual store and memory access handling for
__sanitizer_unaligned_store callbacks to enable tail calling
in MemoryAccess.

Depends on D107282.

Reviewed By: vitalybuka, melver

Differential Revision: https://reviews.llvm.org/D107283

item of StringTable.

Summary: For the string table in XCOFF, the first 4 bytes
contains the length of the string table, so we should
print the string entries from fifth bytes. This patch
also adds tests for llvm-readobj dumping the string
table.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D105522

Add AccessVptr access type.
For now it's converted to the same thr->is_vptr_access,
but later it will be passed directly to ReportRace
and will enable efficient tail calling in MemoryAccess function
(currently __tsan_vptr_update/__tsan_vptr_read can't use
tail calls in MemoryAccess because of the trailing assignment
to thr->is_vptr_access).

Depends on D107276.

Reviewed By: vitalybuka, melver

Differential Revision: https://reviews.llvm.org/D107282

Currently we have MemoryAccess function that accepts
"bool kAccessIsWrite, bool kIsAtomic" and 4 wrappers:
MemoryRead/MemoryWrite/MemoryReadAtomic/MemoryWriteAtomic.

Such scheme with bool flags is not particularly scalable/extendable.
Because of that we did not have Read/Write wrappers for UnalignedMemoryAccess,
and "true, false" or "false, true" at call sites is not very readable.

Moreover, the new tsan runtime will introduce more flags
(e.g. move "freed" and "vptr access" to memory acccess flags).
We can't have 16 wrappers and each flag also takes whole
64-bit register for non-inlined calls.

Introduce AccessType enum that contains bit mask of
read/write, atomic/non-atomic, and later free/non-free,
vptr/non-vptr.
Such scheme is more scalable, more readble, more efficient
(don't consume multiple registers for these flags during calls)
and allows to cover unaligned and range variations of memory
access functions as well.

Also switch from size log to just size.
The new tsan runtime won't have the limitation of supporting
only 1/2/4/8 access sizes, so we don't need the logarithms.

Also add an inline thunk that converts the new interface to the old one.
For inlined calls it should not add any overhead because
all flags/size can be computed as compile time.

Reviewed By: vitalybuka, melver

Differential Revision: https://reviews.llvm.org/D107276

I'm renaming the flag because a future patch will add a new
enableOrderedReductions() TTI interface and so the meaning of this
flag will change to be one of forcing the target to enable/disable
them. Also, since other places in LoopVectorize.cpp use the word
'Ordered' instead of 'strict' I changed the flag to match.

Differential Revision: https://reviews.llvm.org/D107264

These instructions have an implicit use of vcc which counts towards the
constant bus limit. Pre gfx10 this means that the explicit operands
cannot be sgprs. Use the custom inserter hook to call legalizeOperands
to enforce that restriction.

Fixes https://bugs.llvm.org/show_bug.cgi?id=51217

Differential Revision: https://reviews.llvm.org/D106868

Don't just check for the existence of one, but check which one was
specified last, if any.

This fixes https://llvm.org/PR51296.

Differential Revision: https://reviews.llvm.org/D107261

This does the same fix as D107237 but for a couple more options,
converting all remaining cases of such options to accept both
forms, for consistency. This fixes building e.g. openldap, which
uses --image-base=<value>.

Differential Revision: https://reviews.llvm.org/D107253

Reviewed By: hctim

Differential Revision: https://reviews.llvm.org/D107270

Add new pass LowerRefTypesIntPtrConv to generate debugtrap
instruction for an inttoptr and ptrtoint of a reference type instead
of erroring, since calling these instructions on non-integral pointers
has been since allowed (see ac81cb7e).

Differential Revision: https://reviews.llvm.org/D107102

Silence clang-tidy warning in AffineOps.cpp due to the inability to see
through the typeswitch. NFC.

Differential Revision: https://reviews.llvm.org/D106125

In OMP 5.0 specification clause-list with
* task_reduction
* allocate
were allowed on taskgroup construct.

Fix XFAIL - omp-taskloop01.f90.

Reviewed By: kiranchandramohan

Differential Revision: https://reviews.llvm.org/D93373

[RISCV] Teach VSETVLI insertion to merge the unused VSETVLI with the one need to be insert after it.

If a vsetvli instruction is not compatible with the next vector instruction,
and there is no other things that may update or use VL/VTYPE, we could merge
it with the next vsetvli instruction that should be insert for the vector
instruction.

This commit only merge VTYPE with the former vsetvli instruction which has
the same VL.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D106857

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D106865

In RISCV's relocations, some relocations are comprised of two relocation types. For example, R_RISCV_PCREL_HI20 and R_RISCV_PCREL_LO12_I compose a PC relative relocation. In general the compiler will set a label in the position of R_RISCV_PCREL_HI20. So, to test the R_RISCV_PCREL_LO12_I relocation, we need decode instruction at position of the label points to R_RISCV_PCREL_HI20 plus 4 (the size of a riscv non-compress instruction).

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D105528

There are cases in which it is not desirable to fully compose the bound map with the operands when adding lower/upper bounds to a `FlatAffineConstraints`.

E.g., this is the case when bounds should be expressed in terms of the operands only (and not the operands' dependencies). This also makes `addLowerOrUpperBound` useable together with operands that are defined through semi-affine expressions.

Differential Revision: https://reviews.llvm.org/D107221

Bounds such as `dim_{pos} <= c_1 * dim_x + ...` where `x == pos` are invalid. `addLowerOrUpperBound` previously added an incorrect inequality to the set. Such cases are now explicitly rejected.

Differential Revision: https://reviews.llvm.org/D107220

Add ForLoopBoundSpecialization pass, which specializes scf.for loops into a "main loop" where `step` divides the iteration space evenly and into an scf.if that handles the last iteration.

This transformation is useful for vectorization and loop tiling. E.g., when vectorizing loads/stores, programs will spend most of their time in the main loop, in which only unmasked loads/stores are used. Only the in the last iteration (scf.if), slower masked loads/stores are used.

Subsequent commits will apply this transformation in the SparseDialect and in Linalg's loop tiling.

Differential Revision: https://reviews.llvm.org/D105804

In order to support different types of perf scripts, this change tried to refactor `PerfReader` by adding the base class `PerfReaderBase` and current HybridPerfReader is derived from it for CS profile generation. Common functions like, passMM2PEvents, extract_lbrs, extract_callstack, etc. can be reused.

Next step is to add LBR only reader(for non-CS profile) and aggregated perf scripts reader(do a pre-aggregation of scripts).

Reviewed By: hoy, wenlei

Differential Revision: https://reviews.llvm.org/D107014

This is to fix the assert @bjope reported due to the code change of https://reviews.llvm.org/D106589. The test case from @bjope is also included.

Reviewed By: efriedma

Differential Revision: https://reviews.llvm.org/D107302

This adds handling for two cases:

1. A scalable vector where the element type is promoted.
2. A scalable vector where the element count is odd (or more generally,
   not divisble by the element count of the part type).

(Some element types still don't work; for example, <vscale x 2 x i128>,
or <vscale x 2 x fp128>.)

Differential Revision: https://reviews.llvm.org/D105591

The declaration for the global new function in C++ is generated in the compiler front-end. When examining exception propagation, we found that this is the largest root throw site propagator requiring unwind code to be generated for callers up the stack. Allowing this to be handled immediately with termination stops upward propagation and leads to significantly less landing pads generated. This in turns leads to a performance and .text size win.

With `-fnew-infallible` this annotates the declaration with `throw()` and `__attribute__((returns_nonnull))`.  `throw()` allows the compiler to assume exceptions do not propagate out of new and eliminate it as a root throw site. Note that the definition of global new is user-replaceable so users should ensure that the one used follows these semantics.

Measuring internally, we're seeing at 0.5% CPU win in one of our large internal FB workload. Measuring on clang self-build (cd0a1226) we get:

thinlto/

        "dwarfehprepare.NumCleanupLandingPadsRemaining": 153494,
        "dwarfehprepare.NumNoUnwind": 26309,
thinlto_newinfallible/

        "dwarfehprepare.NumCleanupLandingPadsRemaining": 143660,
        "dwarfehprepare.NumNoUnwind": 28744,

a 1-143660/153494 = 6.4% reduction in landing pads and a 28744/26309 = 9.3% increase in the number of nounwind functions.

Testing:
ninja check-all
new test case to make sure these attributes are added correctly to global new.

Reviewed By: urnathan

Differential Revision: https://reviews.llvm.org/D105225

... and rename it to 'warnIfNonZero' to better-reflect what it actually
does.

The goal is to minimize the amount of logic that's conditionally
compiled under '#if __APPLE__'.