This links to the same information but in a new location.

Missed HasRoundModeOp makes performCombineVMergeAndVOps use wrong operands for
VFCVT_RM instructions.

Reviewed By: luke

Differential Revision: https://reviews.llvm.org/D158711

When SelectiondDAG converts dbg.value intrinsics, it first ensures we have
already generated code for the value operator of the intrinsic. The rationale
being that if we haven't had the need to generate code for this value, it won't
be a debug value that causes the generation.

For example, if the first use the physical register of an argument is a
dbg.value, we are going to hit this code path.  However, this is irrelevant for
entry value expressions: by definition we are not interested in the _current_
value of the physical register, but rather on its value at the start of the
function. To deal with this, this patch changes lowering to handle this case as
early as possible.

Differential Revision: https://reviews.llvm.org/D158649

This is partial revert of cbca9ce9.
That commit removed the code guarding against min/max SPF patterns,
because those are now canonicalized to min/max intrinsics. However,
this is only true for integer min/max, while FP min/max can not
always be canonicalized to an intrinsic. As such, restore a
simplified version of the guard that handles only the FP case.

Fixes https://github.com/llvm/llvm-project/issues/64937.

This dependency was introduced by D158638. It seems harmless to add this, as the
Analysis library also does it.

Differential Revision: https://reviews.llvm.org/D158729

Differential Revision: https://reviews.llvm.org/D155406

Differential Revision: https://reviews.llvm.org/D155422

When resolving a frame index with a large offset for v6M execute-only,
we emit a tMOVimm32 pseudo-instruction, which later gets lowered to a
sequence of instructions, all of which are flag-setting. However, a
frame index may be generated for a register spill or reload instruction,
which can be inserted at a point where CPSR is live. This patch inserts
MRS and MSR instructions around the tMOVimm32 to save and restore the
value of CPSR, if CPSR is live at that point.

This may need up to two virtual registers (one to build the immediate
value, one to save CPSR) during frame index lowering, which happens
after register allocation, so we need to ensure two spill slots are
avilable to the register scavenger to ensure it can free up enough
registers for this.

There is no test for the emission (or not) of the MRS/MSR pair, because
it requires a spill or reload to be inserted at a point where CPSR is
live, which requires a large, complex function and is fragile enough
that any optimisation changes will break the test. This bug was easily
found by csmith with -verify-machineinstrs, which I now run regularly on
v6M execute-only (and many other combinations).

Patch by John Brawn and myself.

Reviewed By: stuij

Differential Revision: https://reviews.llvm.org/D158404

Disabled linking by adding -c flag

Reviewed By: kiranchandramohan

Differential Revision: https://reviews.llvm.org/D158727

Only X86_64 and ARM64 have a reserved register for async arguments, and so the
debugger is only able to handle those targets. For other architectures, we use a
non-entry-value expression and let the debugger do its best with that.

Differential Revision: https://reviews.llvm.org/D158638

We should also test the x86 target, since it has different backend defaults from
ARM.

Differential Revision: https://reviews.llvm.org/D158636

My previous patch D151283 added a new option to llvm-cov to make it generate a hierarchical report for better view. This patch is to enable it for the coverage report of LLVM itself.

Reviewed By: phosek

Differential Revision: https://reviews.llvm.org/D158703

Suppose we have a nested loop like this:

  void foo(int32_t *dst, int32_t *src, int m, int n) {
    for (int i = 0; i < m; i++) {
      for (int j = 0; j < n; j++) {
        dst[(i * n) + j] += src[(i * n) + j];
      }
    }
  }

We currently generate runtime memory checks as a precondition for
entering the vectorised version of the inner loop. However, if the
runtime-determined trip count for the inner loop is quite small
then the cost of these checks becomes quite expensive. This patch
attempts to mitigate these costs by adding a new option to
expand the memory ranges being checked to include the outer loop
as well. This leads to runtime checks that can then be hoisted
above the outer loop. For example, rather than looking for a
conflict between the memory ranges:

1. &dst[(i * n)] -> &dst[(i * n) + n]
2. &src[(i * n)] -> &src[(i * n) + n]

we can instead look at the expanded ranges:

1. &dst[0] -> &dst[((m - 1) * n) + n]
2. &src[0] -> &src[((m - 1) * n) + n]

which are outer-loop-invariant. As with many optimisations there
is a trade-off here, because there is a danger that using the
expanded ranges we may never enter the vectorised inner loop,
whereas with the smaller ranges we might enter at least once.

I have added a HoistRuntimeChecks option that is turned off by
default, but can be enabled for workloads where we know this is
guaranteed to be of real benefit. In future, we can also use
PGO to determine if this is worthwhile by using the inner loop
trip count information.

When enabling this option for SPEC2017 on neoverse-v1 with the
flags "-Ofast -mcpu=native -flto" I see an overall geomean
improvement of ~0.5%:

SPEC2017 results (+ is an improvement, - is a regression):
520.omnetpp: +2%
525.x264: +2%
557.xz: +1.2%
...
GEOMEAN: +0.5%

I didn't investigate all the differences to see if they are
genuine or noise, but I know the x264 improvement is real because
it has some hot nested loops with low trip counts where I can
see this hoisting is beneficial.

Tests have been added here:

  Transforms/LoopVectorize/runtime-checks-hoist.ll

Differential Revision: https://reviews.llvm.org/D152366

Could use more work for vectors.

https://reviews.llvm.org/D156534

Currently we're getting away with post-selection constant folding on
these (a hack which exists for the DAG).

https://reviews.llvm.org/D156534

Dst is never used after creating the type making
these assignments dead

Reviewed by: arsenm
Differential Revision: https://reviews.llvm.org/D158610

When lowering a splat_vector_parts, if the hi bits are undefined then we can
splat the lo bits without having to check if it's going to be sign extended or
not, because those bits will be undefined anyway.

I've handled it for both fixed and scalable vectors, but there's no diff
on the scalable vror tests, since the hi bits aren't combined away to
undef in SimplifyDemanded for scalable vectors. I'm not sure why that is.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D158625

This adds some more extensive test coverage for fneg through global isel,
switching the opcodes to use the more complete ActionDefinitions to handle more
cases.

Differential Revision: https://reviews.llvm.org/D154075

Reviewed By: arsenm
Differential Revision: https://reviews.llvm.org/D156865

[X86] fold-and-shift-x86_64.ll - add zext test case where upper bits are known zero (and won't get simplified to any_extend)

Add test coverage showing failure to use foldMaskAndShiftToScale with zero_extend nodes

When visiting load and store instructions in SROA skip scalable vectors.
This is relevant in the implementation of the 'arm_sve_vector_bits'
attribute that is used to define VLS types, similar to D85725.

Fix https://gcc.godbolt.org/z/o561P9zj4

Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D158631

This adds some more extensive test coverage for fmul through global isel,
switching the opcodes to use the more complete ActionDefinitions to handle more
cases.

`reifyResultShapes` should return an `Attribute` if and only if the respective dimension is static.

This fixes #64256.

Differential Revision: https://reviews.llvm.org/D158166

The uaddlv with v8i8 returns 16-bits value but clang generates 32-bits intrinsic
and trunc for it. In this case, we can mark known zero for the high 16-bits of
the intrinsic output.

Differential Revision:

For consistency with other tests and to simplify the `RUN` lines, switch
to using `mlir-cpu-runner` instead of `lli` in integrations tests
targeting SSVE and SME.

Differential Revision: https://reviews.llvm.org/D158719

Split an index register from IndexReg = SIGN_EXTEND(ADD_NSW(X,C)) to IndexReg = SIGN_EXTEND(X), Offset = SIGN_EXTEND(C)

The change introduces intrinsics 'get_fpmode', 'set_fpmode' and
'reset_fpmode'. They manage all target dynamic floating-point control
modes, which include, for instance, rounding direction, precision,
treatment of denormals and so on. The intrinsics do the same
operations as the C library functions 'fegetmode' and 'fesetmode'. By
default they are lowered to calls to these functions.

Two main use cases are supported by this implementation.

1. Local modification of the control modes. In this case the code
usually has a pattern (in pseudocode):

    saved_modes = get_fpmode()
    set_fpmode(<new_modes>)
    ...
    <do operations under the new modes>
    ...
    set_fpmode(saved_modes)

In the case when it is known that the current FP environment is default,
the code may be shorter:

    set_fpmode(<new_modes>)
    ...
    <do operations under the new modes>
    ...
    reset_fpmode()

Such patterns appear not only in user code but also in implementations
of various FP controlling pragmas. In particular, the implementation of
`#pragma STDC FENV_ROUND` requires similar code if the target does not
support static rounding mode.

2. Portable control of FP modes. Usually FP control modes are set by
writing to some control register. Different targets have different
layout of this register, the way the register is accessed also may be
different. Using set of target-specific definitions for the control
register bits together with these intrinsic functions provides enough
portable way to handle control modes across wide range of hardware.

This change defines only llvm intrinsic function, which implement the
access required for the aforementioned use cases.

Differential Revision: https://reviews.llvm.org/D82525

[X86] foldMaskAndShiftToScale - use isShiftedMask_64 directly instead of separate LZ/TZ counting logic

I've updated foldMaskedShiftToBEXTR as well to use matching code.

Option to express that `spirv` StorageClasses should be mapped to LLVM
address spaces in the conversion process. This mapping will be
client-dependent.

The client API cannot be taken from the code as more than one module
could be present, resulting in more than one VCE triple and different
StorageClass to address space mappings. This information would not be
available during type conversion.

A specific mapping for the OpenCL client is defined, based on [the
OpenCL Extended Instruction
Set](https://registry.khronos.org/SPIR-V/specs/unified1/OpenCL.ExtendedInstructionSet.100.html#_binary_form)
and [this
mapping](https://github.com/llvm/llvm-project/blob/3edd338a6407d9410f6a283c5dc32ba676ac0b8f/clang/lib/Basic/Targets/SPIR.h#L27

).

Signed-off-by: Victor Perez <victor.perez@codeplay.com>

Reviewed By: antiagainst, kuhar

Differential Revision: https://reviews.llvm.org/D158627

Annotate constructor/destructor names as FunctionDeclarationName.

Fixes #63046.

Differential Revision: https://reviews.llvm.org/D157963

Differential Revision: https://reviews.llvm.org/D158571

ArmSME is also 'Scalable' but since these tests are predicated on
different configuration flags they can't live in the same directory, so
it makes sense to rename this directory to ArmSVE. This is also
consistent with Vector CPU integration tests.

Reviewed By: awarzynski

Differential Revision: https://reviews.llvm.org/D158712

We no longer allow calls to functions with the `amdgpu_gfx` calling
convention from functions with the `amdgpu_cs_chain_preserve` calling
convention. See D153517.

Also mention that we can't have a chain call from
amdgpu_cs_chain_preserve using more VGPRs than it has received.

Differential Revision: https://reviews.llvm.org/D156408

This reverts commit 16ccba51.

This is failing across Linaro's bots e.g.:
https://lab.llvm.org/buildbot/#/builders/188/builds/34393