Fixes #64229.

Differential Revision: https://reviews.llvm.org/D156655

The refactored template can now be used with MachineVerifier.

Resubmitted after fixing build errors:

- Shared libraries build failed with undefined references due to "extern
  template" declarations.
- Modules build failed due to a cycle dependence between llvm/ADT and llvm/IR.
  The Generic*Impl.h files should be in llvm/IR to prevent this.

Differential Revision: https://reviews.llvm.org/D156522

This restores commit 93a37067.
Originally reverted in 466bd998.

isOperationLegalOrCustomOrPromote returns true only if VT is other or legal
and operation action is Legal, Custom or Promote.
Permit a vector binary operation can be converted to scalar binary operation which is custom lowered with illegal type.
One of cases is i32 isn't a legal type on RV64 and its ALU operations is set to custom lowering,
so vadd for element type i32 can be converted to addw.

Reviewed By: jacquesguan, craig.topper

Differential Revision: https://reviews.llvm.org/D156692

Delay consuming tokens until we are certain that the next token is not top
level block. Otherwise we bail out as if we saw an @end for better diagnostic
and recovery.

Fixes https://github.com/llvm/llvm-project/issues/64065.

Reviewed By: rjmccall

Differential Revision: https://reviews.llvm.org/D156277.

Enforce the following restriction specified in 2.13
A var may appear at most once in all the clauses of declare directives for a
function, subroutine, program, or module.

Reviewed By: razvanlupusoru

Differential Revision: https://reviews.llvm.org/D156945

Lower the deviceptr clause for the OpenACC declare directive.

Reviewed By: razvanlupusoru

Differential Revision: https://reviews.llvm.org/D156828

This change exposes the reversebits library function for HLSL, excluding floating point types.
The reversebits function is supported for all scalar, vector, and matrix types.

The full documentation of the HLSL reversebits function is available here:
https://learn.microsoft.com/en-us/windows/win32/direct3dhlsl/reversebits

Reviewed By: python3kgae

Differential Revision: https://reviews.llvm.org/D156933

fp-imm.ll and zfh-imm.ll test 0.0 and -0.0 while float/double/half-imm.ll
tested other non-zero constants. It seems like they should all be
tested together.

There are slight coverage changes due to different command lines,
but I'm not sure its meaningful. For example, we now don't test
double 0.0 and -0.0 with only the F extension.

Reviewed By: asb

Differential Revision: https://reviews.llvm.org/D156929

D155929 teach lowerScalarInsert to handl start value (extractelement scalable_vector, 0)
and specifically converts fixed extracted vectors to scalable vectors when
lowering vector reduction. It's not enough because there is another way to
create (extractelement fixed_vector, 0) as a start value of lowerScalarInsert
like #64327.

#64327: https://github.com/llvm/llvm-project/issues/64327.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D156863

Fixes #64322

Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D156855

The Fuchsia zxtest library has ASSERT_DEATH but not EXPECT_DEATH.
The latter may be added in the future, but for now just use the
former as substitute.

Reviewed By: abrachet

Differential Revision: https://reviews.llvm.org/D156940

This op is the batched version of linalg.mmt4d. It performs matrix-matrix-transpose multiplication of batched 4-d (5d) inputs as the following:

```
C[b, m1, n1, m0, n0] = sum_{b, k1, k0}(A[b, m1, k1, m0, k0] * B[b, n1, k1, n0, k0])
```

The current use is to provide `linalg.batch_matmul` a lowering path similar to `linalg.matmul -> linalg.mmt4d`.

Differential Revision: https://reviews.llvm.org/D156912

This patch adds patterns for the existing riscv_shl_vl VL node.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D156915

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D156941

We haven't supported JIT on AIX yet.

Fix AIX buildbot failure in https://lab.llvm.org/buildbot/#/builders/214/builds/8770.

Reviewed By: Jake-Egan

Differential Revision: https://reviews.llvm.org/D156921

Lower the copyout clause for the OpenACC declare directive

Depends on D156738

Reviewed By: razvanlupusoru

Differential Revision: https://reviews.llvm.org/D156824

The OpenACC 3.3 specification does not allow the `zero` modifier
on the `copyout` clause used with the declare directive.

This is similar to D156703 for the create clause.

This might be missing piece in the spec but we disallow it until proven otherwise.

Reviewed By: razvanlupusoru

Differential Revision: https://reviews.llvm.org/D156825

This reverts commit e77e891d.

Differential Revision: https://reviews.llvm.org/D156947

Lots of users use "po" as their default print command. If the type
doesn't implement the description function the output is often not what
the user wants. Print a hint telling the user that they might prefer
using "p" instead.

Differential Revision: https://reviews.llvm.org/D153489

For code like:
struct foo { ... };
struct bar { struct foo foo; };
const struct foo my_foo = { ... };
struct bar my_bar = { .foo = my_foo };

Eli Friedman points out the relevant part of the C standard seems to
have some flexibility in what is considered a constant expression:

6.6 paragraph 10:
An implementation may accept other forms of constant expressions.

GCC 8 added support for these, so clang not supporting them has been a
constant thorn in the side of source code portability within the Linux
kernel.

Fixes: https://github.com/llvm/llvm-project/issues/44502

Reviewed By: efriedma

Differential Revision: https://reviews.llvm.org/D76096

We previously defaulted to `sm_35` for the purpose of unspecified
architecture. This was removed in new CUDA versions so we should bump
this up.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D156936

The first trivial example I tried failed to merge due to the user scan
logic. Remove the complicated scan of users handling with distance
thresholds, with a same block restriction. The actual expansion of
sincos is basically the same size as sin or cos individually. Copy the
technique the generic optimization uses, which is to just use the
input instruction as the insert point or just insert at the start of
the entry block.

https://reviews.llvm.org/D156706

Ran across this when making a change to RISCV memset lowering. Seems very odd that manually merging a store into a vector prevents it from being further merged.

Differential Revision: https://reviews.llvm.org/D156349

Return gracefully instead of crashing. Add missing type conversion
tests.

Fixes: https://github.com/llvm/llvm-project/issues/61044

Reviewed By: qedawkins

Differential Revision: https://reviews.llvm.org/D156942

We currently spell check options that are listed as unsupported, but
this doesn't make much sense. If an option is explicitly unsupported
why would one that's spelled similarly be useful?

It looks like the reason this was added was that we explicitly mark
all `--something` flags as Unsupported rather than just leaving them
undefined and treating them as unknown. Drop that handling so that we
don't regress on things like misspelling `--help`.

Differential Revision: https://reviews.llvm.org/D156925

Check the interrupt flag while interpreting IR expressions and allow the
user to interrupt them.

Differential revision: https://reviews.llvm.org/D156822

On Fedora, rocminfo is a fedora package and rocm_agent_enumberator is
installed to /usr/bin.  This causes this error when building.

CMake Error at external/llvm-project/mlir/lib/ExecutionEngine/CMakeLists.txt:232 (message):
  Could not run rocm_agent_enumerator and ROCM_TEST_CHIPSET is not defined

So use find_program() to look for rocm_agent_enumerator instead of assuming a
single location.

Signed-off-by: Tom Rix <trix@redhat.com>

Reviewed By: krzysz00

Differential Revision: https://reviews.llvm.org/D156826

This reverts commit 8b76b44e.

SUMMARY OF CHANGES
------------------
This patch aims to reduce test duplication and to improve code re-use in
SparseTensor integration tests for CPU. This is a direct follow-up of:
  1. https://reviews.llvm.org/D155403 (test duplication), and
  2. https://reviews.llvm.org/D155405 (code re-use),

The key logic for this patch is implemented in:
  * SparseTensor/CPU/lit.local.cfg.
Essentially, the set-up that used to be repeated across all test files
has been extracted into a common LIT configuration file. This makes code
re-use straightforward.

All SVE/VLA tests are now enabled _conditionally_ and refactored to use
`mlir-cpu-runner` rather than `lli`. The former helps with test
duplication and the latter with code re-use.

A few additional refactoring changes are included.

1. The reduce verbosity, long runtime library names like:

  %mlir_native_utils_lib_dir/libmlir_c_runner_utils%shlibext

are replaced with:

  %mlir_c_runner_utils

2. In order to keep the code and the comments in sync, and to maintain
   consistency across the tests, the following:

  enable-runtime-library=true

is swapped with (and vice-versa):

  enable-runtime-library=false

Note that this change won't affect test coverage. Only few tests
required such update.

3. A VLS vectorization `RUN` line is added in tests where there was a
   VLA/VLS `RUN` line, but no VLS `RUN` line (with a few exceptions of
   tests that only contained one `RUN` line to begin with).

4. A few test variables are renamed/added. Most notable example:
  * %{options}` --> %{sparse_compiler_opts}

TEST RUNTIME IMPROVEMENT
------------------------
Tl;Dr This change improves test execution time by ~25%.

At the moment, the following `llvm-lit` invocation takes ~7.30s on my
AArch64 workstation (with SVE):

  llvm-lit  <llvm-project>/mlir/test/Integration/Dialect/SparseTensor/CPU/

This timing doesn't change no matter what the value of the following
CMake variable is (that should disable some tests):

  MLIR_RUN_ARM_SVE_TESTS

With this patch, the execution time will indeed depend on the value of
the above CMake variable:
  * with `MLIR_RUN_ARM_SVE_TESTS=true` the timing remains intact,
  * with `MLIR_RUN_ARM_SVE_TESTS=false` the timing drops to ~5.40s (~25%
    improvement).
This is expected:
  * on average there are 4 `RUN` lines per test,
  * _without this change_ (and with `MLIR_RUN_ARM_SVE_TESTS=false`) the
    4th `RUN` line would in most cases duplicate the 3rd `RUN` line,
  * _with this change) (and with `MLIR_RUN_ARM_SVE_TESTS=false`) the
    4th `RUN` line becomes empty.

PATCH SIZE
----------
While rather large and touching many files, most changes in this patch
are rather mechanical. All test configurations have been preserved and
only in a handful of cases new `RUN` lines added.

Differential Revision: https://reviews.llvm.org/D156625

Two run lines fail due to missing device libs for gfx902, which are fixed by
using gfx1010 which has device lib.

The other two spack tests have to be removed since there is no reliable
way to make them pass/fail not depending on whether there is ROCm
installed on the system.

Reviewed by: Fangrui Song, Joseph Huber

Differential Revision: https://reviews.llvm.org/D156935

* Allow pull request for clang-format changes

Remove repo lockdown on clang-format specific directories

* Missing / from unittests

"Flags" usually refers to boolean options and is used as such in other
places of Options.td (e.g. "Target-dependent compilation options",
"<clang-cl ignored options>").
This patch changes some misnomer group DocName from "flags" to
"options".

Other libc implementations support underscores in NaN(n-char-sequence)
strings. Us not supporting that is causing fuzz failures, so this patch
solves the problem.

Reviewed By: lntue

Differential Revision: https://reviews.llvm.org/D156927

CodeGen/X86/pseudo_cmov_lower2.ll fails using libc++ debug mode
(D150264) without this change.

Reviewed By: MaskRay, aeubanks

Differential Revision: https://reviews.llvm.org/D155811

Breaks CSKY tests.

This reverts commit 948f205a.

This patch fixes failing tests after checking the return code from the
driver. This is mostly due to the ROCm libraries not being present
during most compilations. Passing `-nogpuinc` should allow us to compile
without it for tests that require it. Additionally, some old tests set
the architecture of Nvidia tests to `sm_35` which is officially
unsupported in CUDA 12+ so it prints an error. We just increase in this
case.

Reviewed By: MaskRay, yaxunl

Differential Revision: https://reviews.llvm.org/D156930

Revert "Reapply: [MemCpyOpt] implement single BB stack-move optimization which unify the static unescaped allocas"""

Breaks Asan and LTO.

This reverts commit ea72b513.