Differential Revision: https://reviews.llvm.org/D143271

The bots report:
LLVM ERROR: MachO doesn't support COMDATs, 'f' cannot be lowered.

[AMDGPU] Modify adjustInliningThreshold to also consider the cost of passing function arguments through the stack

A regression from when new PM got enabled as default. Functions with a big number of instructions will elide getting inlined but do not consider the cost of passing arguments over stack if there are a lot of function arguments. This patch attempts to add a heuristic for AMDGPU's function calling convention that also considers function arguments passed through the stack.

Reviewed By: #amdgpu, arsenm

Differential Revision: https://reviews.llvm.org/D140242

The revision uses tablegen to convert multiple atomic and comparison
related enums automatically rather than using hand coded functions
in the import and export from and to LLVM IR.

The revision also adds additional binary operation cases to the
AtomicBinOp enum that have not been supported till now. It also
introduces the possibility to define unsupported enum cases that exist
only in LLVM IR and that are not imported into MLIR. These unsupported
cases are helpful to handle sentinel values such as BAD_BINOP that
LLVM commonly uses to terminate its enums.

Reviewed By: Dinistro

Differential Revision: https://reviews.llvm.org/D143189

This reverts commit 60dcc70e.

This breaks builders, e.g.
https://lab.llvm.org/buildbot/#/builders/36/builds/30036

Symbol/Header::all() calls

Just pop the pointer.

This NFC (intended) patch has several small changes:
* It renames PredicationStyle to TailFoldingStyle.
* It renames TTI.emitActiveLaneMask() to TTI.getPreferredTailFoldingStyle()
* Simplifies some of its uses in the LoopVectorizer

Rationale: To my surprise PredicationStyle::None did not mean 'no
predication', but rather 'no active lane mask intrinsic', such that the
predicate is created using a splat + compare with stepvector. The enum is
also highly specific to tail folding, so it seems better to name this
around that feature, i.e. 'tail folding style'.

This also makes it more amenable to extend it to other tail folding styles,
such as the one added in D142109.

Reviewed By: david-arm

Differential Revision: https://reviews.llvm.org/D142887

ValueTracking attempts to match compare+select patterns to FP min/max
operations, but it was created before the newer IEEE-754-2019
minimum/maximum ops were defined. Ie, matchSelectPattern() does not
account for the -0.0/+0.0 behavior that is specified in the newer
standard.

FMINIMUM/FMAXIMUM nodes were created to map to the newer standard:

/// FMINIMUM/FMAXIMUM - NaN-propagating minimum/maximum that also treat -0.0
/// as less than 0.0. While FMINNUM_IEEE/FMAXNUM_IEEE follow IEEE 754-2008
/// semantics, FMINIMUM/FMAXIMUM follow IEEE 754-2018 draft semantics.

We could adjust ValueTracking to deal with signed zero, but it seems like
a moot point given the divergent NaN behavior discussed in D143056, so just
delete this possibility to avoid bugs when converting IR to SDAG.

Differential Revision: https://reviews.llvm.org/D143106

This test is supposed to work in arm64.

Differential Revision: https://reviews.llvm.org/D143265

These were using ” ("right double quotation mark") instead of
the usual ". This means CMake took the value literally and you
got:
```
CMake Error at CMakeLists.txt:139 (MESSAGE):
  ”libc” isn't a known project:
  bolt;clang;clang-tools-extra;compiler-rt;cross-project-tests;libc;libclc;lld;lldb;mlir;openmp;polly;pstl;flang.
  Did you mean to enable it as a runtime in LLVM_ENABLE_RUNTIMES?
```
`="libc"` works fine.

Reviewed By: lntue

Differential Revision: https://reviews.llvm.org/D143254

With the current set of opcodes, this saves 3460 lines in the generated
Opcodes.inc in release builds (-17%).

Differential Revision: https://reviews.llvm.org/D142694

For atomics metadata, we can make data race analysis more efficient by
entirely ignoring functions that include memory accesses but which only
access non-escaping (non-shared) and/or non-mutable memory. Such
functions will not be considered to be covered by "atomics" metadata,
resulting in the following benefits:

  1. reduces "covered" metadata; and
  2. allows data race analysis to skip such functions.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D143159

Adapted from the existing shuffle-of-fneg tests.

The original implementation of the transform interpreter pass base was
cloning the entire transform IR in presence of PDL-related operations to
avoid concurrency issues when running the pass with the same transform
IR on multiple operations of the payload IR. The root cause of those
issues is the `transform.pdl_match` operation that was moving the PDL
pattern definition operation into a new module, consumed by the PDL
interpreter and leading to a race. Clone the pattern operation instead.
This avoids the race as well as the cost for transform IR that doesn't
use PDL.

Depends on D142729.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D142962

The transform dialect infrastructure does not provide a default
interpreter pass and instead expects users to create their own to ensure
all relevant extensions and dependent dialects are loaded. Provide a
base class for implementing such passes that includes the additional
facilities for debugging and is aware of the multithreaded nature of
pass execution.

Reviewed By: pifon2a, nicolasvasilache

Differential Revision: https://reviews.llvm.org/D142729

Implement MaterializeTemporaryExpr for primitive types.

Differential Revision: https://reviews.llvm.org/D136017

The code did not propagate the result optionality for subroutine.
Make the result of genIntrinsicRef optional.

Differential Revision: https://reviews.llvm.org/D143251

The current behavior for AMX macros is:

```
gcc -march=native -dM -E - < /dev/null | grep TILE

clang -march=native -dM -E - < /dev/null | grep TILE
```

which is not ideal.  Change `__AMXTILE__` and friends to `__AMX_TILE__` (i.e.
have an underscore in them).  This makes GCC and Clang agree on the naming of
these AMX macros to simplify downstream user code.

Fix this for `__AMXTILE__`, `__AMX_INT8__`, `__AMX_BF16__`, and `__AMX_FP16__`.

Differential Revision: https://reviews.llvm.org/D143094

While we potentially need to align partially mapped structs more than
the first member, we do not need to align past the struct itself. This
prevents us from moving the base pointer past the struct beginning too.

See https://reviews.llvm.org/D142508 for a discussion.

Reviewed By: pavelkopyl, grokos, jhuber6

Differential Revision: https://reviews.llvm.org/D142586

This reverts commit d65c3bf3.

Differential Revision: https://reviews.llvm.org/D141858

Inspired by Issue #60464 - wel can add/enable additional intrinsics when we add expansion support

`llvm.load` op has nonTemporal field which is missing for `memref.load` and `memref.store`. This revision first adds nonTemporal field to memref's load/store op, then it lowers the field to llvm.load/store ops.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D142616

These address some remaining reasons to #include StdSymbolMap.inc directly.

Differential Revision: https://reviews.llvm.org/D142467

Differential Revision: https://reviews.llvm.org/D143179

This patch adds the LLVM IR intrinsics for the following:

* sdot (2-way, vectors + indexed)
* udot (2-way, vectors + indexed)
* fdot (vectors + indexed)

Differential Revision: https://reviews.llvm.org/D143066

genRecordAssignment is emitting code to call Assign in the runtime for some cases.
In these cases, the finalization is done by the runtime so we do not need to do it in
a separate cal to avoid multiple finalization..
Also refactor the code in Bridge so the actual finalization of allocatable
is done before any reallocation. We might need to push this into ReallocIfNeeded.
It is not clear if the allocatable lhs needs to be finalized in any cases or only if it is
reallocated.

Reviewed By: jeanPerier

Differential Revision: https://reviews.llvm.org/D143186

I committed an older version of the patch without the test updates.
This patch uses the latest versions on
https://reviews.llvm.org/D142904

This patch adds the LLVM IR intrinsics for the following:

* zip (2 and 4 vectors, 8 to 64-bit elements) - aarch64.sve.zip.*
* zip (2 and 4 vectors, 128-bit elements) - aarch64.sve.zipq.*
* uzp (2 and 4 vectors, 8 to 64-bit elements) - aarch64.sve.uzp.*
* uzp (2 and 4 vectors, 128-bit elements) - aarch64.sve.uzpq.*

I have created separate intrinsics for the 128-bit variants
in a similar way to what was done for int_aarch64_sme_readq_horiz.
This permits us to use any vector type (<vscale x 16 x i8>, etc.)
for the 128-bit versions.

I have also named the tests sve2p1-intrinsics-* because although
the instructions are added as part of the SME2 feature they only
operate on SVE vectors.

NOTE: These intrinsics are still in development and are subject to future changes.

Differential Revision: https://reviews.llvm.org/D142904

Second instance.

Promote offset to 32-bit, similarily to what D142549 did for flat_offset.

Differential Revision: https://reviews.llvm.org/D143174

isSafeToSpeculativelyExecute always return true for casts, so remove
this redundant checks.

Differential Revision: https://reviews.llvm.org/D143255

According to the IR verifier, "Declaration[s] may not be in a Comdat!"

This is a re-commit of 76b3f0b4 with
updates to the test:
 * Force emission of the extra-module, to trigger the bug after D138264,
   by providing a second symbol @g, and making the comdat nodeduplicate.
   (Technically only one is needed, but two should be safer.)
 * Name the comdat $f to avoid failure on Windows:
   LLVM ERROR: Associative COMDAT symbol 'c' does not exist.

Differential Revision: https://reviews.llvm.org/D142443

This patch fixes #60307 issue. The 8bb4451a introduces the possibility
to unite overlapped or adjacent address ranges to keep address ranges
in an unambiguous state. The AddressRangesMap is used to normalize
address ranges. The AddressRangesMap keeps address ranges and the value
of the relocated address. For intersected range, it creates a united
range that keeps the last inserted mapping value. The same for adjusted ranges.
While it is OK to use the last inserted mapping value for intersected ranges
(as there is no way how to resolve ambiguity) It is not OK to use the
last inserted value for adjacent address ranges. Currently, two following
address ranges are united into a single one:

{0,24,17e685c} {24,d8,55afe20} -> {0,d8,55afe20}

To avoid the problem, the AddressRangesMap should not unite adjacent address ranges
with different relocated addresses. Instead, it should leave adjacent address ranges
as separate ranges. So, the ranges should look like this:

{0,24,17e685c} {24,d8,55afe20}

Differential Revision: https://reviews.llvm.org/D142936