Wrong type was used for the result type in the tosa.conv_2d canonicalization.
The type should match the result element type should match the result type
not the input element type.

Differential Revision: https://reviews.llvm.org/D115463

G_PTR_ADD takes arguments of two different types, so it probably shouldn't be
considered commutative just on that basis. A recent G_PTR_ADD reassociation
optimization (https://reviews.llvm.org/D109528) can emit erroneous code if the
pattern matcher commutes the arguments; this can happen when the base pointer
was created by G_INTTOPTR of a G_CONSTANT and the offset register is variable.

This was discovered on the llvm-mos fork, but I added a failing test case that
should apply to AArch64 (and more generally).

Differential Revision: https://reviews.llvm.org/D114655

This change adds options to llvm-ifs to allow it to generate multiple
types of stub files at a single invocation.

Differential Revision: https://reviews.llvm.org/D115024

The test has been flaky for years, and I think we should remove it to
eliminate noise on the buildbot.

Neither me nor dokyungs have been able to fully deflake the test, and it
tests a non-default Entropic flag.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D115453

This removes the last use of StackDepot from StopTheWorld.

Depends on D115284.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D115319

val can be of any type accepted by Compare.

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D115364

Necessary for implementing some combines on floating point selects.

Differential Revision: https://reviews.llvm.org/D115372

StackDepot locks some stuff. As is there is small probability to
deadlock if we stop thread which locked the Depot.

We need either Lock/Unlock StackDepot for StopTheWorld, or don't
interact with StackDepot from there.

This patch does not run LeakReport under StopTheWorld. LeakReport
contains most of StackDepot access.

As a bonus this patch will help to resolve kMaxLeaksConsidered FIXME.

Depends on D114498.

Reviewed By: morehouse, kstoimenov

Differential Revision: https://reviews.llvm.org/D115284

This reapplies a fix from 948ce4e6,
whichn't originally submitted upstream. I has now been merged upstream
though, in https://github.com/google/benchmark/pull/1302.

When benchmarks were unified in
5dda2efd, it lost this change,
but it also lost another local modification, where benchmark's
CMakeLists.txt was modified to comment out adding -Werror.
(This change was part of the original import in
0addd170.)

As the benchmark library is built automatically by default, when
building all of LLVM (contrary to the copy in libcxx, which wasn't
built by default), building it with -Werror by default is very brittle.

This fixes building LLVM with MinGW. (It wasn't broken in MSVC
mode, as the benchmark library doesn't add -Werror or anything
equivalent in MSVC mode, and it's unclear if this warning is
enabled in that mode at all.)

Differential Revision: https://reviews.llvm.org/D115434

Reapply #2 of [runtimes] Fix building initial libunwind+libcxxabi+libcxx with compiler implied -lunwind

This does mostly the same as D112126, but for the runtimes cmake files.
Most of that is straightforward, but the interdependency between
libcxx and libunwind is tricky:

Libunwind is built at the same time as libcxx, but libunwind is not
installed yet. LIBCXXABI_USE_LLVM_UNWINDER makes libcxx link directly
against the just-built libunwind, but the compiler implicit -lunwind
isn't found. This patch avoids that by adding --unwindlib=none if
supported, if we are going to link explicitly against a newly built
unwinder anyway.

Since the previous attempt, this no longer uses
llvm_enable_language_nolink (and thus doesn't set
CMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY during the compiler
sanity checks). Setting CMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY
during compiler sanity checks makes cmake not learn about some
aspects of the compiler, which can make further find_library or
find_package fail. This caus...

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D115318

The comparator for the sort functions should provide strict weak
ordering relation between parameters. Current solution causes compiler
crash with some standard c++ library implementations, because it does
not meet this criteria. Tried to fix it + it improves the iverall
vectorization result.

Differential Revision: https://reviews.llvm.org/D115268

The code claimed to handle nsw/nuw, but those aren't passed via builder state and the explicit IR construction just above never sets them.

The only case this bit of code is actually relevant for is FMF flags.  However, dropPoisonGeneratingFlags currently doesn't know about FMF at all, so this was a noop.  It's also unneeded, as the caller explicitly configures the flags on the builder before this call, and the flags on the individual ops should be controled by the intrinsic flags anyways.  If any of the flags aren't safe to propagate, the caller needs to make that change.

Just a simple typo fix that allows me to test landing a commit now that
I have commit access.

Reviewed By: xgupta

Differential Revision: https://reviews.llvm.org/D115414

The recurrence lowering code has handling which claims to be about flag intersection, but all the callers pass empty arrays to the arguments.  The sole exception is a caller of a method which has the argument, but no implementation.

I don't know what the intent was here, but it certaintly doesn't actually do anything today.

As in D114934, or lsan crashes on the same bot.

Depends on D114495.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D114498

This reorders existing transforms to put demanded elements last. The reasoning here is that when we have an example which can be scalarized or handled via demanded bits, we should prefer scalarization as that doesn't require dropping flags on arithmetic instructions.

This doesn't show major changes in the tests today, but once I add support for fast math flags to dropPoisonGeneratingFlags this becomes glaringly obvious.

Differential Revision: https://reviews.llvm.org/D115394

This reverts commit e5c2a46c as this
change introduced a linker error when building sanitizer runtimes:

  ld.lld: error: undefined symbol: __sanitizer::internal_start_thread(void* (*)(void*), void*)
  >>> referenced by sanitizer_stackdepot.cpp:133 (compiler-rt/lib/sanitizer_common/sanitizer_stackdepot.cpp:133)
  >>>               compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommonSymbolizer.x86_64.dir/sanitizer_stackdepot.cpp.obj:(__sanitizer::(anonymous namespace)::CompressThread::NewWorkNotify())

This change allows us to estimate trip count from profile metadata for all multiple exit loops. We still do the estimate only from the latch, but that's fine as it causes us to over estimate the trip count at worst.

Reviewing the uses of the API, all but one are cases where we restrict a loop transformation (unroll, and vectorize respectively) when we know the trip count is short enough. So, as a result, the change makes these passes strictly less aggressive. The test change illustrates a case where we'd previously have runtime unrolled a loop which ran fewer iterations than the unroll factor. This is definitely unprofitable.

The one case where an upper bound on estimate trip count could drive a more aggressive transform is peeling, and I duplicated the logic being removed from the generic estimation there to keep it the same. The resulting heuristic makes no sense and should probably be immediately removed, but we can do that in a separate change.

This was noticed when analyzing regressions on D113939.

I plan to come back and incorporate estimated trip counts from other exits, but that's a minor improvement which can follow separately.

Differential Revision: https://reviews.llvm.org/D115362

Transform
```
(~a & b & c) | ~(a | b | c) -> ~(a | (b ^ c))
```
And swapped case:
```
(~a | b | c) & ~(a & b & c) -> ~a | (b ^ c)
```

```
----------------------------------------
define i4 @src(i4 %a, i4 %b, i4 %c) {
%0:
  %or1 = or i4 %b, %a
  %or2 = or i4 %or1, %c
  %not1 = xor i4 %or2, 15
  %not2 = xor i4 %a, 15
  %and1 = and i4 %b, %not2
  %and2 = and i4 %and1, %c
  %or3 = or i4 %and2, %not1
  ret i4 %or3
}
=>
define i4 @tgt(i4 %a, i4 %b, i4 %c) {
%0:
  %1 = xor i4 %c, %b
  %2 = or i4 %1, %a
  %or3 = xor i4 %2, 15
  ret i4 %or3
}
Transformation seems to be correct!
```
```
----------------------------------------
define i4 @src(i4 %a, i4 %b, i4 %c) {
%0:
  %and1 = and i4 %b, %a
  %and2 = and i4 %and1, %c
  %not1 = xor i4 %and2, 15
  %not2 = xor i4 %a, 15
  %or1 = or i4 %not2, %b
  %or2 = or i4 %or1, %c
  %and3 = and i4 %or2, %not1
  ret i4 %and3
}
=>
define i4 @tgt(i4 %a, i4 %b, i4 %c) {
%0:
  %xor = xor i4 %b, %c
  %not = xor i4 %a, 15
  %or = or i4 %xor, %not
  ret i4 %or
}
Transformation seems to be correct!
```

Differential Revision: https://reviews.llvm.org/D112966

This change applies two fixes to the abseil-cleanup-ctad check. It uses hasSingleDecl() to ensure only declStmt()s with one varDecl() are matched (leaving compount declStmt()s unchanged). It also addresses a bug in the handling of comments that surround the absl::MakeCleanup() calls by switching to the callArgs() combinator from Clang Transformer.

Reviewed By: ymandel

Differential Revision: https://reviews.llvm.org/D115452

This patterns tries to convert an inner (outer) dim reduction to an
outer (inner) dim reduction. Doing this on a 1D or 0D vector results
in an infinite loop since the converted op is same as the original
operation. Just returning failure when source rank <= 1 fixes the
issue.

Differential Revision: https://reviews.llvm.org/D115426

Some of the function name in the comment didn't match
their actual name in the runtime. This patch fixes that.

Reviewed By: schweitz

Differential Revision: https://reviews.llvm.org/D115076

This reverts commit 5a33e412 becuase it
breaks LLDB.

https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/39208/

D113805 improved handling of i32 divu/remu on RV64. The basic idea
from that can be extended to (mul (and X, C2), C1) where C2 is any
mask constant.

We can replace the and with an SLLI by shifting by the number of
leading zeros in C2 if we also shift C1 left by XLen - lzcnt(C1)
bits. This will give the full product XLen additional trailing zeros,
putting the result in the output of MULHU. If we can't use ANDI,
ZEXT.H, or ZEXT.W, this will avoid materializing C2 in a register.

The downside is it make take 1 additional instruction to create C1.
But since that's not on the critical path, it can hopefully be
interleaved with other operations.

The previous tablegen pattern is replaced by custom isel code.

Reviewed By: asb

Differential Revision: https://reviews.llvm.org/D115310

lldb.cpp is unused after https://github.com/llvm/llvm-project/commit/ccf1469a4cdb03cb2bc7868f76164e85d90ebee1

Differential Revision: https://reviews.llvm.org/D115438

Reviewed By: gchatelet

Differential Revision: https://reviews.llvm.org/D115370

-Remove feq, fle, flt tests from *-arith.ll in favor of *-fcmp.ll which tests all predicates.

Reviewed By: asb

Differential Revision: https://reviews.llvm.org/D113703

The sparse tensor code generator allocates memory for the output tensor. As
such, we only need to allocate a MemRefDescriptor to receive the output tensor
and do not need to allocate and initialize the storage for the tensor.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D115292

This patch adds the documentation for the operations `omp.atomic.read`,
`omp.atomic.write` and `omp.atomic.update`.

Reviewed By: peixin

Differential Revision: https://reviews.llvm.org/D115445

This patch adds on an overhead cost for gathers and scatters, which
is a rough estimate based on performance investigations I have
performed on SVE hardware for various micro-benchmarks.

Differential Revision: https://reviews.llvm.org/D115143

- Define a gpu.printf op, which can be lowered to any GPU printf() support (which is present in CUDA, HIP, and OpenCL). This op only supports constant format strings and scalar arguments
- Define the lowering of gpu.pirntf to a call to printf() (which is what is required for AMD GPUs when using OpenCL) as well as to the hostcall interface present in the AMD Open Compute device library, which is the interface present when kernels are running under HIP.
- Add a "runtime" enum that allows specifying which of the possible runtimes a ROCDL kernel will be executed under or that the runtime is unknown. This enum controls how gpu.printf is lowered

This change does not enable lowering for Nvidia GPUs, but such a lowering should be possible in principle.

And:
[MLIR][AMDGPU] Always set amdgpu-implicitarg-num-bytes=56 on kernels

This is something that Clang always sets on both OpenCL and HIP kernels, and failing to include it causes mysterious crashes with printf() support.

In addition, revert the max-flat-work-group-size to (1, 256) to avoid triggering bugs in the AMDGPU backend.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D110448

I've added some tests that were previously missing for the gather-scatter costs
being calculated by the vectorizer for AArch64:

  Transforms/LoopVectorize/AArch64/sve-gather-scatter-cost.ll

The costs are sometimes different to the ones in

  Analysis/CostModel/AArch64/sve-gather.ll

because the vectorizer also adds on the address computation cost.

This reverts commit 543a9ad7.

Depends On D115263

By aligning block size to inner loop iterations parallel_compute_fn LLVM can later unroll and vectorize some of the inner loops with small number of trip counts. Up to 2x speedup in multiple benchmarks.

Reviewed By: bkramer

Differential Revision: https://reviews.llvm.org/D115436