The problem was whitespace between the comment and the code for the
matcher. Rather than fix the script, I went the easier route and
removed the offending newline. If this problem comes up again though,
we should consider making the script less fragile.

This bit-hack transform would cause the new test to infinite loop
after 21d3871b.

The deleted transform has existed for a very long time,
but the profitable parts appear to be handled by other
folds now. This fold could replace 2 instructions with
4 instructions, so it was always in danger of going
overboard.

No tests regress by removing the whole thing.

Note, when regenerating the documentation for this change, no changes
were made to the HTML file. There is an existing bug with the AST
matcher python script that fails to handle hasBody() specifically. So
this commit is to correct the internal documentation but another change
will be needed to fix the public documentation.

Also regenerates the documentation and fixed a validation diagnostic
about use of 'is' vs '=='.

When `strict = true`, only pre-existing and newly-created ops are rewritten and/or folded. Such ops are stored in `strictModeFilteredOps`.

Newly-created ops were previously added to `strictModeFilteredOps` after calling `addToWorklist` (via `GreedyPatternRewriteDriver::notifyOperationInserted`). Therefore, newly-created ops were never added to the worklist.

Also fix a test case that should have gone into an infinite loop (`test.replace_with_new_op` was replaced with itself, which should have caused the op to be rewritten over and over), but did not due to this bug.

Differential Revision: https://reviews.llvm.org/D141141

Tweaks elements of the new API for filtering the set of modeled fields.

Differential Revision: https://reviews.llvm.org/D141319

Android only provides this function on API 28+; fix libc++ builds when
targeting older API levels.

Reviewed By: arichardson, #libc, philnik

Differential Revision: https://reviews.llvm.org/D141184

Reinstate use of FakeSection class to avoid constructing SectionChunk
from unintialised coff_section in FakeSectionChunk constructor.

Issue was caused by commit 5a58b19f,
"[LLD] Remove global state in lld/COFF".

Merges `TransferOptions` into the newly-introduced
`DataflowAnalysisContext::Options` and removes explicit parameter for
`TransferOptions`, relying instead on the common options carried by the analysis
context. Given that there was no intent to allow different options between calls
to `transfer`, a common value for the options is preferable.

Differential Revision: https://reviews.llvm.org/D140703

Just specify the encoded bytes instead, which causes less confusion anyway.

Differential Revision: https://reviews.llvm.org/D141368

There are a lot of variants of atomic operations, and AArch64 has several
distinct options to codegen them, and does different things depending on
available features, architecture version and optimisation level. The current
testing for atomic operations has been added gradually over time and does not
give full coverate.  Given how complex the codegen for atomic operations is, it
is valuable to cover the entire parameter space, i.e. test them all. The
resulting set of tests serve also as a reference for how each is codegened.

In order to keep the test files readable and avoid constant updating for
unrelated codegen changes, the test outputs are filtered to only include the
relevant instructions. This shows for each operation and feature which codegen
approach is taken (e.g. ll/sc loop, atomic instruction, library call).

The following parameter space is tested:
 - feature: +lse, +rcpc, etc
 - optimisation level: O0, O1 (covers GISel and SelectionDAG)
 - atomic instruction: load, store, cmpxchg, atomirmw*
 - size: i8, i16, i32, i64, i128
 - aligned/unaligned accesses
 - endianness: big, little
 - atomic ordering: release, acquire, etc
 - load atomic only: const/non-const
 - cmpxchg only: weak/strong
 - atomicrmw: update operation (add, sub, etc)

Notably not covered:
 - volatility: there should be no difference between volatile/non-volatile
 - atomicrmw fadd/fsub

The files are split by triple, LLVM instruction, and feature. This makes it
easy to diff between features and endianness for a given instruction.

The file that generates the tests is included.

There are 70 test files with an average of 2800 lines each.

Differential Revision: https://reviews.llvm.org/D141064

Add a verifier to the TransformOpInterface ensuring that operations
implementing the interface define memory effects on their operands and
results.

Add the missing effects to TileToForeachThreadOp, specifically for
operands that were added at a later version of the op without modifying
`getEffects` accordingly.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D141371

This reverts commit 313b747c.
It wasn't reviwed as yet.

Extract a static method to call the MLIR builders that translate LLVM IR
instructions to MLIR LLVM dialect operations. This change ensures the
MLIR builders have to use the moduleImport argument rather than calling
the import methods directly. As a result, both the intrinsic and the
instruction MLIR builders have to use a moduleImport argument and none
of them has direct access to private moduleImport methods. The revision
thus enforces consistent MLIR builder implementations for instructions
and intrinsics.

The revision also moves parseDataLayoutAlignment closer to its use.

Reviewed By: Dinistro

Differential Revision: https://reviews.llvm.org/D141265

Just specify the encoded bytes instead, which causes less confusion anyway.

Differential Revision: https://reviews.llvm.org/D141312

Differential Revision: https://reviews.llvm.org/D141362

As a result, we have identical alignment calculation of byval
parameters for:

  - LowerCall() - getting alignment of an argument (.param)

  - emitFunctionParamList() - getting alignment of a
    parameter (.param) in a function declaration

  - getPrototype() - getting alignment of a parameter (.param) in a
    function prototypes that is used for indirect calls

This change is required to avoid ptxas error: 'Alignment of argument
does not match formal parameter'. This error happens even in cases
where it logically shouldn't.

For instance:

  .param .align 4 .b8 param0[4];
  ...
  callprototype ()_ (.param .align 2 .b8 _[4]);
  ...

Here we allocate 'param0' with alignment of 4 and it should be fine to
pass it to a function that requires minimum alignment of 2.

At least ptxas v12.0 rejects this code.

Differential Revision: https://reviews.llvm.org/D140581

Support of variadic functions triggers an assertion on several tests
from llvm/test/CodeGen/Generic/ if nvptx64-* is specified as a default
triplet:

Support for variadic functions (unsized array parameter) introduced in
PTX ISA version 6.0 and requires target sm_30.

That happens because those tests contain variadic function calls and
default versions of both PTX ISA (3.2) and architecture (sm_20) are
below the minimally required.

There were no observable problems with these tests before adding
support of variadic functions, because nvptx backend just didn't
handle them properly generating invalid PTX code.

Differential Revision: https://reviews.llvm.org/D141054

Keeping bitcasts in the common_bitcast() test, otherwise sinking
no longer occurs there. GVNSink seems to have a silly cost model
where sinking just the stores is considered non-profitable, but
sinking the stores and bitcasts (which are free...) is profitable.

As this is not a default-enabled pass, and this problem would exist
without opaque pointers as well, I'm not trying to do anything
about this.

When feature CSSC is available, the abs intrinsic should map to the
new scalar ABS instruction when using GlobalIsel

spec:
https://developer.arm.com/documentation/ddi0602/2022-09/Base-Instructions/ABS--Absolute-value-

Reviewed By: aemerson

Differential Revision: https://reviews.llvm.org/D139419

Avoid use of no-op GEPs.

As far as I can see they are unnecessary and in C++20 they lead to errors.
Removing them the tests pass with C++20 and the default version.
Example error message:
```
llvm-project/llvm/unittests/IR/PassBuilderCallbacksTest.cpp:275:30: error: expected unqualified-id before ‘)’ token
  275 |   MockAnalysisHandle<Module>() { setDefaults(); }
```

Differential Revision: https://reviews.llvm.org/D141358

The GreedyPatternRewriteDriver did previously not count the first iteration. I.e., when setting `config.maxIterations = 1`, two iterations were performed. In pratice, this number is not really important; we usually just need a limit in some reasonable order of magnitude. However, this fix allows us to write better convergence/worklist tests with carefully crafted test patterns to purposely trigger edge cases in the driver.

Similarly, the first rewrite was previously not counted towards `config.maxNumRewrites`.

For consistency, `OpPatternRewriteDriver` now uses `config.maxNumRewrites` instead of `config.maxIterations`; this driver does not have "iterations", it consists of a single loop (corresponding to the inner loop in the GreedyPatternRewriteDriver).

Differential Revision: https://reviews.llvm.org/D141365

Reapply with checks for instructions in unreachable blocks. A test
case for this was added in 1ee4a93b.

-----

This is a recurring pattern: We want to find the nearest common
dominator (instruction) for two instructions, but currently only
provide an API for the nearest common dominator of two basic blocks.

Add an overload that accepts and return instructions.

This only effects folks building with wasm64 + shared memory which
is not currently a supported configuration in emscripten or any other
wasm toolchain.

Differential Revision: https://reviews.llvm.org/D141005

wasm32 was already supported, so here I've just reused the same regex. I'm not sure if this is actually correct: I don't know for certain if wasm32 and wasm64 have different output formats, but it seems to work.

Reviewed By: dschuff, asb

Differential Revision: https://reviews.llvm.org/D141130

This reverts commit 957952db.

Addition in the newly added code can overflow.  As a result, the
constructor of `BranchProbability()` can trigger an assertion. See
the discussion on https://reviews.llvm.org/D138132 for more details.

This is a follow-up to https://reviews.llvm.org/D140896, split into
several parts as it touches a lot of files.

Differential Revision: https://reviews.llvm.org/D141298