Instead of using a WHILE XZR, XZR instruction, just emit a PFALSE.

Reviewed By: david-arm

Differential Revision: https://reviews.llvm.org/D109311

This patch changes the default monotonicity of dynamic schedule from
monotonic to non-monotonic when no modifier is specified.

Differential Revision: https://reviews.llvm.org/D109026

We previously didn't have any tests to defend the cost model
for gathers and scatters using SVE without a vscale_range
attribute. I've added tests to existing files:

  Analysis/CostModel/AArch64/sve-gather.ll
  Analysis/CostModel/AArch64/sve-scatter.ll

Differential Revision: https://reviews.llvm.org/D109055

Move inverse_throughput, latency and uops to sub-directories (like we already do for lbr), which require libpfm, so we can relax the lit limits for analysis tests in the x86 root directory.

Differential Revision: https://reviews.llvm.org/D109353

Add KnownBits handling and unit tests for X*X self-multiplication cases which guarantee that bit1 of their results will be zero - see PR48683.

https://alive2.llvm.org/ce/z/NN_eaR

The next step will be to add suitable test coverage so this can be enabled in ValueTracking/DAG/GlobalISel - currently only a single Analysis/ScalarEvolution test is affected.

Differential Revision: https://reviews.llvm.org/D108992

Legalize G_MEMCPY, G_MEMMOVE, G_MEMSET and G_MEMCPY_INLINE.

Corresponding intrinsics are replaced by a loop that uses loads/stores in
AMDGPULowerIntrinsics pass unless their length is a constant lower then
MemIntrinsicExpandSizeThresholdOpt (default 1024). Any G_MEM* instruction that
reaches legalizer should have a const length argument and should be expanded
into appropriate number of loads + stores.

Differential Revision: https://reviews.llvm.org/D108357

This patch adds support for the vector-predicated `VP_STORE` and
`VP_LOAD` nodes. We do this in the same way we lower `MSTORE` and
`MLOAD`: to regular load/store instructions via intrinsics.

One necessary change was made to `SelectionDAGLegalize` so that
`VP_STORE` nodes' operation actions are taken from the stored "value"
operands, in the same vein as `STORE` or `MSTORE`.

Reviewed By: craig.topper, rogfer01

Differential Revision: https://reviews.llvm.org/D108999

This patch adds support for the `VP_SCATTER` and `VP_GATHER` nodes by
lowering them to RVV's `vsox`/`vlux` instructions, respectively. This
process is almost identical to the existing `MSCATTER`/`MGATHER` support.

One extra change was made to `SelectionDAGLegalize` so that
`VP_SCATTER`'s operation action is derived from its stored "value"
operand rather than its return type (which is always the chain).

Reviewed By: craig.topper, rogfer01

Differential Revision: https://reviews.llvm.org/D108987

In the case of no tied variables, we pick random defs, and then random uses that don't alias with defs we just picked.
Sounds good, except that an X86 instruction may have implicit reg uses,
e.g. for `MULX` it's `EDX`/`RDX`: `Intel SDM, 4-162 Vol. 2B MULX — Unsigned Multiply Without Affecting Flags`
> Performs an unsigned multiplication of the implicit source operand (EDX/RDX) and the specified source operand
> (the third operand) and stores the low half of the result in the second destination (second operand), the high half
> of the result in the first destination operand (first operand), without reading or writing the arithmetic flags.

And indeed, every once in a while `llvm-exegesis` happened to pick EDX as a def while measuring throughput,
and producing garbage output:
```
$ ./bin/llvm-exegesis -num-repetitions=1000000 -mode=inverse_throughput -repetition-mode=min --loop-body-size=4096 -dump-object-to-disk=false -opcode-name=MULX32rr --max-configs-per-opcode=65536
---
mode:            inverse_throughput
key:
  instructions:
    - 'MULX32rr EDX R11D R12D'
  config:          ''
  register_initial_values:
    - 'R12D=0x0'
    - 'EDX=0x0'
cpu_name:        znver3
llvm_triple:     x86_64-unknown-linux-gnu
num_repetitions: 1000000
measurements:
  - { key: inverse_throughput, value: 4.00014, per_snippet_value: 4.00014 }
error:           ''
info:            instruction has no tied variables picking Uses different from defs
assembled_snippet: 415441BC00000000BA00000000C4C223F6D4C4C223F6D4C4C223F6D4C4C223F6D4415CC3415441BC00000000BA0000000049B80200000000000000C4C223F6D4C4C223F6D44983C0FF75F0415CC3
...
```
```
$ ./bin/llvm-exegesis -num-repetitions=1000000 -mode=inverse_throughput -repetition-mode=min --loop-body-size=4096 -dump-object-to-disk=false -opcode-name=MULX32rr --max-configs-per-opcode=65536
---
mode:            inverse_throughput
key:
  instructions:
    - 'MULX32rr R13D EDX ECX'
  config:          ''
  register_initial_values:
    - 'ECX=0x0'
    - 'EDX=0x0'
cpu_name:        znver3
llvm_triple:     x86_64-unknown-linux-gnu
num_repetitions: 1000000
measurements:
  - { key: inverse_throughput, value: 3.00013, per_snippet_value: 3.00013 }
error:           ''
info:            instruction has no tied variables picking Uses different from defs
assembled_snippet: 4155B900000000BA00000000C4626BF6E9C4626BF6E9C4626BF6E9C4626BF6E9415DC34155B900000000BA0000000049B80200000000000000C4626BF6E9C4626BF6E94983C0FF75F0415DC3
...
```
Oops! Not only does that not look fun, i did hit that pitfail during AMD Zen 3 enablement.
While i have since then addressed this in rGd4d459e7,
i suspect there may be other buggy results lying around, so we should at least stop producing them.

Reviewed By: courbet

Differential Revision: https://reviews.llvm.org/D109275

It is disallowed in OpenCL C to declare static kernel functions and
C++ for OpenCL is expected to inherit such behaviour. Error is now
correctly reported in C++ for OpenCL when declaring a static kernel
function.

Differential Revision: https://reviews.llvm.org/D109150

When expanding pseudo insts, in order to create a new machine instr, we use BuildMI,
which will add implicit operands by default. And transferImpOps will also copy implicit
operands from old ones. Finally, duplicate implicit operands are added to the same inst.
Sometimes this can cause correctness issues. Like below inst,
    renamable $w18 = nsw SUBSWrr renamable $w30, renamable $w14, implicit-def dead $nzcv
After expanding, it will become
    $w18 = SUBSWrs renamable $w13, renamable $w14, 0, implicit-def $nzcv, implicit-def dead $nzcv
A redundant implicit-def $nzcv is added, but the dead flag is missing.

Reviewed By: dmgreen

Differential Revision: https://reviews.llvm.org/D109069

Found while working on D108987. When interpreting VP nodes as
`MemSDNode` nodes, this function would return the incorrect indices.
This was due to `VP_GATHER` and having no "passthru", and both
`VP_GATHER` and `VP_SCATTER` having their mask operands *after* the base
pointer, unlike `MGATHER` and `MSCATTER`.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D109308

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D109292

Differential Revision: https://reviews.llvm.org/D109346

Prevent the folding in DAGCombine if it leads to worse code.

Reviewed By: dmgreen

Differential Revision: https://reviews.llvm.org/D109124

Reviewed By: RKSimon, dmgreen

Differential Revision: https://reviews.llvm.org/D109123

Differential Revision: https://reviews.llvm.org/D108975

Differential Revision: https://reviews.llvm.org/D108972

Using std::vector<DeviceTy> requires implementing copy constructor and copied assign operator for DeviceTy.
Indeed DeviceTy should never be copied. After changing to std::vector<std::unique_ptr<DeviceTy>>,
All the unsafe copy constructor and copy assign operator implementations can be removed.
Compilers mark them deleted due to mutex or underlying objects and this is the desired behavior.

Differential Revision: https://reviews.llvm.org/D109276

Before this patch, we only support syntax like
`clang -cc1 -ast-dump -ast-dump-filter main a.c`
or
`clang -Xclang -ast-dump -Xclang -ast-dump-filter -Xclang main a.c`
when using ast-dump-filter.

It is helpful to also support `-ast-dump-filter=` syntax, so we can do
something like
`clang -cc1 -ast-dump -ast-dump-filter=main a.c`
or
`clang -Xclang -ast-dump -Xclang -ast-dump-filter=main a.c`

It is more cleaner when passing arguments through `-Xclang` in this case.

Also, **clang-check** do support this syntax, and I think people might
be confiused when they found they can't use `ast-dump-filter` with
clang.

Plugins always return int32_t. Stay consistent with other functions which return error status.

Differential Revision: https://reviews.llvm.org/D109341

UsedTLSStorage is only used in allocateTLSSection,
guarded in x87 ELF only.
So clang will emit error with -Werror on.

.../llvm/tools/llvm-rtdyld/llvm-rtdyld.cpp:288:12:
error: private field 'UsedTLSStorage' is not used
[-Werror,-Wunused-private-field]
  unsigned UsedTLSStorage = 0;
           ^

Differential Revision: https://reviews.llvm.org/D108270

This is a more general version of D109273. Though it doesn't
peek through bitcasts or rearange broadcasts.

Reviewed By: LuoYuanke

Differential Revision: https://reviews.llvm.org/D109295

Due to a typo, this replaced %x with umax(C1, umin(C2, %x + C3))
rather than umax(C1, umin(C2, %x)). This didn't make a difference
for the existing tests, because the result is only used for range
calculation, and %x will usually have an unknown starting range,
and the additional offset keeps it unknown. However, if %x already
has a known range, we may compute a result range that is too
small.

The test is based on a miscompile example in:
https://llvm.org/PR51321

Differential Revision: https://reviews.llvm.org/D107692

lldb/source/Core/PluginManager.cpp:695:21: warning: moving a temporary object prevents copy elision [-Wpessimizing-move]
      return Status(std::move(ret.takeError()));
                    ^
lldb/source/Core/PluginManager.cpp:695:21: note: remove std::move call here
      return Status(std::move(ret.takeError()));
                    ^~~~~~~~~~               ~

The current IRSimilarityIdentifier does not try to find similarity across blocks, this patch provides a mechanism to compare two branches against one another, to find similarity across basic blocks, rather than just within them.

This adds a step in the similarity identification process that labels all of the basic blocks so that we can identify the relative branching locations. Within an IRSimilarityCandidate we use these relative locations to determine whether if the branching to other relative locations in the same region is the same between branches. If they are, we consider them similar.

We do not consider the relative location of the branch if the target branch is outside of the region. In this case, both branches must exit to a location outside the region, but the exact relative location does not matter.

Reviewers: paquette, yroux

Differential Revision: https://reviews.llvm.org/D106989

As a reminder if someone comes looking to improve iteration or parent
navigation performance of DWARFDebugInfoEntry.

Extend PluginManager::SaveCore() to support saving core dumps
via Process plugins.  Implement the client-side part of qSaveCore
request in the gdb-remote plugin, that creates the core dump
on the remote host and then uses vFile packets to transfer it.

Differential Revision: https://reviews.llvm.org/D101329

Identified with readability-qualified-auto.