This patch fix the re-revert D135927 (which caused a windows build failure) to re-enable parallel for/reduction. It also fix a warning caused by D137442.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D137565

This patch fixes:

  mlir/lib/Dialect/SparseTensor/IR/SparseTensorDialect.cpp:717:48:
  error: comparison of integers of different signs: 'int64_t' (aka
  'long') and 'uint64_t' (aka 'unsigned long')
  [-Werror,-Wsign-compare]

Lower async.func with async.coro and async.runtime operations

- This patch modifies AsyncToAsyncRuntime pass to add lowering async.func ops with coroutine cfg.
Example:

```
async.func @foo() -> !async.value<f32> {
  %cst = arith.constant 42.0 : f32
  return %cst: f32
}
```

After lowering:

```
func.func @foo() -> !async.value<f32> attributes {passthrough = ["presplitcoroutine"]} {
    %0 = async.runtime.create : !async.value<f32>
    %1 = async.coro.id
    %2 = async.coro.begin %1
    cf.br ^bb1
  ^bb1:  // pred: ^bb0
    %cst = arith.constant 4.200000e+01 : f32
    async.runtime.store %cst, %0 : <f32>
    async.runtime.set_available %0 : !async.value<f32>
    cf.br ^bb2
  ^bb2:  // pred: ^bb1
    async.coro.free %1, %2
    cf.br ^bb3
  ^bb3:  // pred: ^bb2
    async.coro.end %2
    return %0 : !async.value<f32>
}
```

Reviewed By: ezhulenev

Differential Revision: https://reviews.llvm.org/D137462

Remove GlobalISel test files that only contained RUN lines running the
code from the SelectionDAG version of the same test.

Differential Revision: https://reviews.llvm.org/D137533

This reverts commit 53d5d340.

This is causing a build failure on the windows mlir bot that was previously hidden by another sparse tensor change that caused failures:

https://lab.llvm.org/buildbot/#/builders/13/builds/28006

Previously the response file expansion code would print the error, but
lld would not exit, which was odd.

lld does response file expansion in the different drivers, but it's also
done in main() first, so it's enough to check there.

By checking for these errors we would have caught when D136090
introduced a bug that made lld print errors for response files which
contained "-rpath @foo".

Differental revision: https://reviews.llvm.org/D137477

Added MathOptionsBase to share fastmath config between different
components. Frontend driver translates LangOptions into MathOptionsBase.
FirConverter configures FirOpBuilder using MathOptionsBase
config passed to it via LoweringOptions.

Depends on D137390

Reviewed By: jeanPerier

Differential Revision: https://reviews.llvm.org/D137391

Reviewed By: jeanPerier, PeteSteinfeld

Differential Revision: https://reviews.llvm.org/D137546

Add the missing include to fix an error when `cmake_push_check_state()`
is called and incidentally the CMakePushCheckState module is not loaded
by any other check running prior to `FindLibEdit.cmake`:

    CMake Error at /var/no-tmpfs/portage/dev-util/lldb-15.0.4/work/cmake/Modules/FindLibEdit.cmake:24 (cmake_push_check_state):
      Unknown CMake command "cmake_push_check_state".
    Call Stack (most recent call first):
      cmake/modules/LLDBConfig.cmake:52 (find_package)
      cmake/modules/LLDBConfig.cmake:59 (add_optional_dependency)
      CMakeLists.txt:28 (include)

Gentoo Bug: https://bugs.gentoo.org/880065

Differential Revision: https://reviews.llvm.org/D137555

As described in issue #58475, we could pass the attributes of pow to sqrt and crash.

Differential Revision: https://reviews.llvm.org/D137454

D137341

Provide FirOpBuilder::setFastMathFlags() to configure FastMathFlags
for the builder. Set FastMathAttr for operations based on FirOpBuilder
configuration via mlir::OpBuilder::Listener.

This is a little bit hacky solution, because we lose the ability
to hook other listeners to FirOpBuilder. There are also potential
issues with OpBuilder::clone() - the hook will be invoked for cloned
operations and will effectively overwrite FastMathAttr with the ones
configured in FirOpBuilder, which should not be happening.
We should teach mlir::OpBuilder about FastMathAttr setup in future.

Reviewed By: jeanPerier, kiranchandramohan

Differential Revision: https://reviews.llvm.org/D137390

GlobalVariable and Function can be available_externally. GlobalAlias is used
similarly. Allowing available_externally is a natural extension and helps
ThinLTO discard GlobalAlias in a non-prevailing COMDAT (see D135427).

For now, available_externally GlobalAlias must point to an
available_externally GlobalValue (not ConstantExpr).

Differential Revision: https://reviews.llvm.org/D137441

Revert "[mlir][sparse] fix sparse tensor rewriting patterns that do not propagate sparse tensor SSA properly."

This reverts commit 70508b61.

This change depends on a reverted change that broke the windows mlir buildbot; reverting to bring remaining mlir bots to green

This reverts commit 83838978.

This broke the windows mlir buildbot: https://lab.llvm.org/buildbot/#/builders/13/builds/27934

https://alive2.llvm.org/ce/z/JxX5in

This change adds a new NVGPU operation that targets the PTX `mma.sp.sync`
instruction variants. A lowering to NVVM is provided using inline
assembly.

Reviewed By: ThomasRaoux, manishucsd

Differential Revision: https://reviews.llvm.org/D137202

I missed this test in d35fcf0e.

This reverts commit 8ddd1ccd to fix
buildbots failures reported in https://lab.llvm.org/buildbot#builders/74/builds/14839

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D137442

This switches wasm intrinsics to use default attributes,
i.e. nofree, nosync, nocallback and willreturn. Especially
willreturn will be required to avoid optimization regressions
in the future.

The attributes are omitted from the trapping fptoi intrinsics
(where I assume trapping is considered well-defined, and as such
these aren't willreturn), the throw/rethrow intrinsics (which
will unwind) and the atomic intrinsics (which aren't nosync).

Differential Revision: https://reviews.llvm.org/D137551

APInt::getSplat() requires that the new size is >= the original
one. If we're loading less than 8 bits, truncate instead.

Fixes https://github.com/llvm/llvm-project/issues/58845.

Use case:
- When block layout is visualized after MBP pass, the basic blocks are labeled in layout order; meanwhile blocks could be numbered in a different order.
- As a result, it's hard to map between the graph and pass output. With this option on, the basic blocks are renumbered in function layout order.

This option is only useful when a function is to be visualized (i.e., when view options are on) to make it debugging only.

Use https://godbolt.org/z/5WTW36bMr as an example:
- As MBP pass output (shown in godbolt output window), `func2` is in a basic block numbered `2` (`bb.2`), and `func1` is in a basic block numbered `3` (`bb.3`);
    `bb.3` is a block with higher block frequency than `bb.2`, and `bb.3` is placed before `bb.2` in the functin layout.
- Use [1] to get the dot graph (graph uploaded in [2]), the blocks are re-numbered.
   - `func1` is in 'if.end' block, and labeled `1` in visualized dot; `func2` is in 'if.then' blocks, and labeled `3` --> the labeled number and bb number won't map.
   - [[ https://github.com/llvm/llvm-project/blob/b5626ae9751f0d82aa04791a21689b289721738e/llvm/lib/CodeGen/MachineBlockFrequencyInfo.cpp#L127 | DOTGraphTraits<MachineBlockFrequencyInfo *>::getNodeLabel ]] is where labeled numbers are based on function layout number, and [[ https://github.com/llvm/llvm-project/blob/a8d93783f37c042ace67069ae4ca6f8fd849c2d0/llvm/include/llvm/Support/GraphWriter.h#L209
 | called by graph writer ]].
        So call 'MachineFunction::RenumberBlocks' would make labeled number (in dot graph) and block number (in pass output) consistent with each other.

[1] `./bin/clang++ -O3 -S -mllvm -view-block-layout-with-bfi=count -mllvm -view-bfi-func-name=_Z9func_loopv -mllvm -print-after=block-placement -mllvm  -filter-print-funcs=_Z9func_loopv test.c`

[2] {F25201785}

Reviewed By: davidxl

Differential Revision: https://reviews.llvm.org/D137467

This patch adds the assembly/disassembly for the following instructions:

st1w: Contiguous store words from vector (128-bit vector elements)
st1d: Contiguous store doublewords from vector (128-bit vector elements)
ld1w: Contiguous load unsigned words to vector (128-bit vector elements)
ld1d: Contiguous load unsigned doublewords to vector (128-bit vector elements)

The reference can be found here:
https://developer.arm.com/documentation/ddi0602/2022-09

Differential Revision: https://reviews.llvm.org/D137245

Splatting the first vector element of the result of a BinOp, where any of the
BinOp's operands are the result of a first vector element splat can be simplified to
splatting the first vector element of the result of the BinOp

Differential Revision: https://reviews.llvm.org/D135876

Replace the quick sort partition method with one that is more similar to the
method used by C++ std quick sort. This improves the runtime for sorting
sk_2005.mtx by more than 10x.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D137290

This patch adds the new FEAT_B16B16 feature as well as the
assembly/disassembly for all of the B16B16 instructions:

bfadd:   BFloat16 floating-point add vectors
bfsub:   BFloat16 floating-point subtract vectors
bfmul:   BFloat16 floating-point multiply vectors
bfclamp: BFloat16 floating-point clamp to minimum/maximum number
bfmax:   BFloat16 floating-point maximum
bfmaxnm: BFloat16 floating-point maximum number
bfmin:   BFloat16 floating-point minimum
bfminnm: BFloat16 floating-point minimum number
bfmla:   BFloat16 floating-point fused multiply-add vectors
bfmls:   BFloat16 floating-point fused multiply-subtract vectors

The reference can be found here:
https://developer.arm.com/documentation/ddi0602/2022-09

Differential Revision: https://reviews.llvm.org/D137321

There can be a difference for MOVDDUPrr but not the load folded broadcast that is purely on Port23

Fixes an old TODO (inherited from SkylakeServer which was fixed at c7662dc3)

Confirmed on Agner + uops.info

Try to simplify comparisons with the smallest normalized value. If
denormals will be treated as 0, we can simplify by using an equality
comparison with 0.

fcmp olt fabs(x), smallest_normalized_number -> fcmp oeq x, 0.0
fcmp ult fabs(x), smallest_normalized_number -> fcmp ueq x, 0.0
fcmp oge fabs(x), smallest_normalized_number -> fcmp one x, 0.0
fcmp ult fabs(x), smallest_normalized_number -> fcmp ueq x, 0.0

The device libraries have a few range checks that look like
this for denormal handling paths.

A future change will try to fold (if input denormals are treated as 0)
fcmp olt fabs(x), smallest_normalized_number -> fcmp oeq x, 0.0
fcmp ult fabs(x), smallest_normalized_number -> fcmp ueq x, 0.0
fcmp oge fabs(x), smallest_normalized_number -> fcmp one x, 0.0
fcmp ult fabs(x), smallest_normalized_number -> fcmp ueq x, 0.0

Move getDebugValueLoc so that it can be accessed from DebugInfo.h for the
Assignment Tracking patch stack and remove redundant parameter Src.

Reviewed By: jryans

Differential Revision: https://reviews.llvm.org/D132357

Gather nodes are vectorized as simply vector of the scalars instead of
relying on the actual node. It leads to the fact that in some cases
we may miss incorrect transformation (non-matching set of scalars is
just ended as a gather node instead of possible vector/gather node).
Better to rely on the actual nodes, it allows to improve stability and
better detect missed cases.

Differential Revision: https://reviews.llvm.org/D135174

deleteAssignmentMarkers(const Instruction *Inst) does exactly as you'd expect -
it deletes any dbg.assign intrinsics linked to Inst.

Reviewed By: jmorse

Differential Revision: https://reviews.llvm.org/D133576

First patch in a series adding MC layer support for SME2.1.

This patch adds the following feature:

sme2p1

The reference can be found here:
https://developer.arm.com/documentation/ddi0602/2022-09

Differential Revision: https://reviews.llvm.org/D137410

Currently call slot optimization may be prevented because the
lifetime markers for the destination only start after the call.
In this case, rather than aborting the transform, we should move
the lifetime.start before the call to enable the transform.

Differential Revision: https://reviews.llvm.org/D135886

Those methods were added long time ago. Now we get the same methods generated by tablegen, so there is no need for duplicates.

Differential Revision: https://reviews.llvm.org/D137544

This was an accidental addition of a non-reviewed change.

This reverts commit f63db915.