ApplyEachOpTrait applies to payload ops associated with its operand
handle one-by-one in order. If a handle is consumed, this usually
indicates that the associated payload ops are erased or rewritten. Add a
check that we don't consume an ancestor payload operation before
consuming its descendant, as the latter is likely to be a dangling
pointer. Transform operations for which this is a legitimate behavior
(i.e., they consume the handle but don't actually erase or rewrite the
payload operation) should implement the interface directly and allow for
repeated handles.

Reviewed By: springerm

Differential Revision: https://reviews.llvm.org/D152510

This patch uses the (de)interleaving intrinsics introduced in
D141924 to handle vectorization of interleaving groups with a
factor of 2 for scalable vectors.

Reviewed By: fhahn, reames

Differential Revision: https://reviews.llvm.org/D145163

This is useful for transformations such as bufferization, which is looking for tensor.extract_slice/insert_slice pairs.

Also fix the documentation of the corresponding tranform op.

Differential Revision: https://reviews.llvm.org/D152455

Set these operands to poison, which might allow folding the phi node,
or reduce the use count of an instruction.

Fix inline asm trampoline type. Some architectures will complain:

<inline asm>:8:41: error: expected STT_<TYPE_IN_UPPER_CASE>, '#<type>', '%<type>' or "<type>"
    8 | .type  __interceptor_trampoline_malloc, @function

Just use %function instead, which is what is also used in
sanitizer_asm.h

Use the default TrackingListener. No need to set up a derived listener just for the test case. This revision is in preparation of a future change that adds a TrackingRewriter infrastructure.

Differential Revision: https://reviews.llvm.org/D152446

This relocation is used for the 21-bit immediate in ADR instructions.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D151305

A TransformRewriter (with attached TrackingListener) will be added to an interface method in a subsequent revision.

Differential Revision: https://reviews.llvm.org/D152426

This is now checked as part of the usual SCEV verification. There
is little value in checking this on each lookup.

These two maps are strictly synchronized nowadays, which was not
the case historically.

This type was removed in D151520, but downstream dependencies use it. By
introducing a temporary allias, we can give these dependencies a chance to
update their uses of the type and give them a chance to migrate without
breakages.

Reviewed By: csigg

Differential Revision: https://reviews.llvm.org/D152509

When looking for replacement ops (`findReplacementOp`) distinguish between "no replacement could be found" and "this op should be dropped from the mapping". The latter case will be utilized in a subsequent revision when a payload op is mapped to a consumed handle.

Differential Revision: https://reviews.llvm.org/D152375

This shows a current problem with G_CTTZ_ZERO_UNDEF result legalizations

Replace the legacy legalizer versions

This is a bit of cleanup before working on logging via stream operator (i.e., `EXPECT_XXX() << ...`).

Reviewed By: sivachandra

Differential Revision: https://reviews.llvm.org/D152503

This function is extracted from `TransformState::applyTransform`.

Differential Revision: https://reviews.llvm.org/D152374

Rework Linux (and *BSD) interceptors to allow for up to 3 (2 for *BSD)
simultaneous interceptors. See code comments for details.

The main motivation is to support new sampling sanitizers (in the spirit
of GWP-ASan), that have to intercept few functions. Unfortunately, the
reality is that there are user interceptors that exist in the wild.

To support foreign user interceptors, foreign dynamic analysis
interceptors, and compiler-rt interceptors all at the same time,
including any combination of them, this change enables up to 3
interceptors on Linux (2 on *BSD).

v2:
* Revert to to the simpler "weak wrapper -(alias)-> __interceptor"
  scheme on architectures that cannot implement a trampoline efficiently
  due to complexities of resolving a preemptible symbol (PowerPC64
  ELFv2 global entry, and i386 PIC).
* Avoid duplicate intercepted functions in gen_dynamic_list.py, due to
  matching __interceptor_X and ___interceptor_X.
* Fix s390 __tls_get_offset.

Reviewed By: dvyukov, MaskRay, vitalybuka

Differential Revision: https://reviews.llvm.org/D151085

Disable diagnostics for neon attributes for GPU-side CUDA compilation.

Test plan: ninja check-all

Differential revision: https://reviews.llvm.org/D152403

There was an overlook to duplicate the unittests for modules. This patch
removes one of this duplication.

Reformat interception macros to be more readable.

NFC.

The required HTMLLogger include file needs python3 to run
resource_bundle.py to bundle all the html/css/js resources. However, if
user sets -DLLVM_INCLUDE_TESTS=OFF, CMake will not find python3 and the
resource bundler will never be executed. This patch set the python3 as a
required build dependency to fix this problem.

Patch By: Avimitin

Differential Revision: https://reviews.llvm.org/D152418

These are no-ops with opaque pointers.

This removes BitCasts from isSource in Type Promotion, as I don't believe they
need to be treated as Sources. They will usually be from floats or hoisted
constants, where constants will be handled already.

This fixes #62513, but didn't otherwise cause any differences in the tests I
ran.

Differential Revision: https://reviews.llvm.org/D152112

Split from D146023

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D152432

rG2eb7cbf9

 added this code, which results in crash for vector
nodes. This patch solves it by skipping for the vector nodes.

Thanks Steve for helping reducing the test case.

Co-authored-by: Steve Merritt <steve.merritt@intel.com>

Reviewed By: goldstein.w.n

Differential Revision: https://reviews.llvm.org/D152492

A follow-up for 64397d8f. Only do verification if VerifyLoopInfo is
set.

This patch provides an alternative implementation to DPP for Scan Computations.

An alternative implementation iterates over all active lanes of Wavefront
using llvm.cttz and performs the following steps:
    1.  Read the value that needs to be atomically incremented using
        llvm.amdgcn.readlane intrinsic
    2.  Accumulate the result.
    3.  Update the scan result using llvm.amdgcn.writelane intrinsic
        if intermediate scan results are needed later in the kernel.

Reviewed By: arsenm, cdevadas

Differential Revision: https://reviews.llvm.org/D147408

- (op (op X, C1), C2) -> (op X, (op C1, C2))
- (op (op X, C1), Y) -> (op (op X, Y), C1)

Some code duplication with the G_PTR_ADD reassociations unfortunately but no
easy way to avoid it that I can see.

Differential Revision: https://reviews.llvm.org/D150230

I found this during the support of modules for clangd. The reason for
the issue is that the iterator returned by noload_lookups is fast-fail
after the lookup table changes by the design of llvm::DenseMap. And
originally the lookup will try to use getAcceptableDecl to filter the
invisible decls. The key point here is that the function
"getAcceptableDecl" wouldn't stop after it find the specific decl is
invisble. It will continue to visit its redecls to find a visible one.
However, such process involves loading decls from external sources,
which results the invalidation.

Note that the use of "noload_lookups" is rare. It is only used in tools
like FixTypos and CodeCompletions. So it is completely fine for the
tranditional compiler. This is the reason why I can't reproduce it by a
lit test.

Instead, we add an argument `isSEWAware` to indicate that the
instruction is SEW-aware. Actually, the only difference is the
name of pseudo instructions.

And we remove postfix `_E` for all classes/multiclasses and remove
argument `sew` since it can be calculated from `log2sew`.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D152428

`!logtwo` was added in D134068.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D152422

We'd like to support -gsplit-dwarf for Windows COFF. It requires to
write Dwo and NonDwo sections to different output streams.The original
implementation is not designed to do that and there can be only one
MCObjectWriter. This patch split WinCOFFObjectWriter to WinCOFFWriter so
that:
  1. WinCOFFObjectWriter can create multiple WinCOFFWriter.
  2. Each WinCOFFWriter can separately collect sections it is interested.
  3. Each WinCOFFWriter can write to it's own output stream.

Reviewed By: skan

Differential Revision: https://reviews.llvm.org/D152340

Fixes #63017

Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D151778

The ABI of BF16 is identical to FP16 rather than i16.

Fixes #62997

Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D151710

I recently discovered that `.profraw` headers are expected to be 8 byte
aligned.
https://github.com/llvm/llvm-project/blob/643ba926c1f618401c86dc37e659df795db2e1a0/llvm/lib/ProfileData/InstrProfReader.cpp#L503-L506

When function entry coverage mode is used, function counters are single
bytes, so it is likely that the size of the counters section is not 8
byte aligned. We can add padding after the counters section to guarantee
this.

Reviewed By: kyulee, gulfem

Differential Revision: https://reviews.llvm.org/D152479