This reverts commit 6fbf0229, restoring
commit bf6ff4fd with a fix for a bot
failure due to a previously unstable iteration order.

Differential Revision: https://reviews.llvm.org/D141077

`QualType::getNonReferenceType()` does the same thing.

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D149744

The `Iteration` value may be -1 which would cause incorrect loop count
when pass the value to buildSqrtNROneConst or buildSqrtNRTwoConst.

Addresses https://github.com/llvm/llvm-project/issues/53353

Reviewed By: RKSimon, pengfei

Differential Revision: https://reviews.llvm.org/D145634

Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D149114

This patch adds a new preheader block the VPlan to place SCEV expansions
expansions like the trip count. This preheader block is disconnected
at the moment, as the bypass blocks of the skeleton are not yet modeled
in VPlan.

The preheader block is executed before skeleton creation, so the SCEV
expansion results can be used during skeleton creation. At the moment,
the trip count expression and induction steps are expanded in the new
preheader. The remainder of SCEV expansions will be moved gradually in
the future.

D147965 will update skeleton creation to use the steps expanded in the
pre-header to fix #58811.

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D147964

Reduces line noise and localises changes needed for volta

Reviewed By: jhuber6

Differential Revision: https://reviews.llvm.org/D149845

This patch implements the mem2reg interfaces for MemRef types. This only supports scalar memrefs of a small list of types. It would be beneficial to create more interfaces for default values before expanding support to more types. Additionally, I am working on an upcoming revision to bring SROA to MLIR that should help with non-scalar memrefs.

Reviewed By: gysit, Mogball

Differential Revision: https://reviews.llvm.org/D149441

Suggested by @jobnoorman
https://reviews.llvm.org/D148767#4317848

This patch adds the necessary hacks to support global constructors and
destructors. This is an incredibly hacky process caused by the primary
fact that Nvidia does not provide any binary tools and very little
linker support. We first had to emit references to these functions and
their priority in D149451. Then we dig them out of the module once it's
loaded to manually create the list that the linker should have made for
us. This patch also contains a few Nvidia specific hacks, but it passes
the test, albeit with a stack size warning from `ptxas` for the
callback. But this should be fine given the resource usage of a common
test.

This also adds a dependency on LLVM to the NVPTX loader, which hopefully doesn't
cause problems with our CUDA buildbot.

Depends on D149451

Reviewed By: tra

Differential Revision: https://reviews.llvm.org/D149527

This patch mostly adapts the existing AMDGPUCtorDtorLoweringPass for use
by the Nvidia backend. This pass transforms the ctor / dtor list into a
kernel call that can be used to invoke those functinos. Furthermore, we
emit globals such that the names and addresses of these constructor
functions can be found by the driver. Unfortunately, since NVPTX has no
way to emit variables at a named section, nor a functioning linker to
provide the begin / end symbols, we need to mangle these names and have
an external application find them.

This work is related to the work in D149398 and D149340.

Reviewed By: tra

Differential Revision: https://reviews.llvm.org/D149451

The code sequence on gfx9 has a lot of useless v_bfi instructions.

Differential Revision: https://reviews.llvm.org/D149840

Left out of D149788 to simplify the diff

Reviewed By: jhuber6

Differential Revision: https://reviews.llvm.org/D149807

[clang-tidy] bugprone-use-after-move: Ctor arguments should be sequenced if ctor call is written as list-initialization.

See

https://timsong-cpp.github.io/cppwp/n4868/dcl.init#list-4

This eliminates a false positive in bugprone-use-after-move; this newly added
test used to be falsely classified as a use-after-move:

```
    A a;
    S3 s3{a.getInt(), std::move(a)};
```

Reviewed By: PiotrZSL

Differential Revision: https://reviews.llvm.org/D148110

This is passed on to Program::createDescriptor, where it is used as a
value for IsConst.

The previous patches added the necessary support for global constructors
used to register tests. This patch enables the AMDGPU target to build
and run the unit tests on the GPU. Currently this only tests the `ctype`
tests, but adding more should be straightforward from here on.

Reviewed By: sivachandra

Differential Revision: https://reviews.llvm.org/D149517

The following pattern is common in the llvm codebase, as well as in downstream projects:
```
llvm::to_vector(llvm::map_range(container, lambda))
```
This patch introduces a shortcut for this called `map_to_vector`.

This template depends on both `llvm/ADT/SmallVector.h` and `llvm/ADT/STLExtras.h`, and since these are both relatively large and do not depend on each other, the `map_to_vector` helper is placed in a new header under `llvm/ADT/SmallVectorExtras.h`. Only a handful of use cases have been updated to use the new helper.

Differential Revision: https://reviews.llvm.org/D145390

The shared code for lowering the sum and product operations in
flang/lib/Optimizer/HLFIR/Transforms/LowerHLFIRIntrinsics.cpp have
been moved into a new class HlfirReductionIntrinsicConverion.

Depends on: D148719

Differential Revision: https://reviews.llvm.org/D149644

Carries out the initial lowering of the product intrinsic into HLFIR
following a similar method to sum, the --use-hlfir-intrinsic-ops flag
in test/Lower/HLFIR/expr-box is set to false so that the tests will
pass until hlfir.product is lowered into fir.call

Depends on: D147624

Differential Revision: https://reviews.llvm.org/D148719

Adds a HLFIR operation for the PRODUCT intrinsic according to
the design set out in flang/doc/HighLevelFIR.md

Since the PRODUCT intrinsic is essentially identical to SUM
in terms of its arguments and result characteristics in the
Fortran Standard, the operation definition and subsequent
tests also take the same form.

Differential Revision: https://reviews.llvm.org/D147624

This commit eliminates the uninitialized error state from the class
RegionRawOffsetV2 (which is locally used by the Clang Static Analyzer
checker alpha.security.ArrayBoundV2) and replaces its use with
std::optional.

Motivated by https://reviews.llvm.org/D148355#inline-1437928

Moreover, the code of RegionRawOffsetV2::computeOffset() is rearranged
to clarify its behavior. The helper function getValue() was eliminated
by picking a better initial value for the variable Offset; two other
helper functions were replaced by the lambda function Calc() because
this way it doesn't need to take the "context" objects as parameters.

This reorganization revealed some surprising (but not outright buggy)
behavior that's marked by a FIXME and will be revisited in a separate
commit.

Differential Revision: https://reviews.llvm.org/D149259

It looks like the intention here is to truncate a XLenVT -> i1, in
which case we should be emitting snez instead of sneq if I'm understanding
correctly.

Reviewed By: jacquesguan, frasercrmck

Differential Revision: https://reviews.llvm.org/D149732

They don't convey any useful information and make the documentation
unnecessarily hard to read.

Differential Revision: https://reviews.llvm.org/D149641

This reverts commit 6a808270.

This test was added after https://reviews.llvm.org/D147506 was comittted which
has been reverted pending a buildbot fix for:
https://lab.llvm.org/buildbot/#/builders/247/builds/4125

Another test based on review comments added late in the review.

This one confirms that the multiplication and addition of the outer
index to the inner index and thus form the 2D index.

Reviewed By: tblah

Differential Revision: https://reviews.llvm.org/D149265

This reverts commit b48a8233.

This change caused https://lab.llvm.org/buildbot/#/builders/247/builds/4125
to start failing, please address the failures before resubmitting.

Adds HSA timeout hint of 2 seconds to the AMDGPU nextgen-plugin to improve
performance of small kernels.
The HSA runtime may stay in HSA_WAIT_STATE_ACTIVE for up to the timeout
value before switching to HSA_WAIT_STATE_BLOCKED. This can improve
latency from which small kernels can benefit.
The value was determined via experimentation w/ different benchmarks.

The timeout value can be overriden using the environment variable
LIBOMPTARGET_AMDGPU_STREAM_BUSYWAIT with a value in microseconds.

Original author: Greg Rodgers <Gregory.Rodgers@amd.com>
Contributions from: JP Lehr <JanPatrick.Lehr@amd.com>

Differential Revision: https://reviews.llvm.org/D148808

The qualifier alignment fixer appeared to ignore any ranges specified for limiting formatting.
This change ensures that it only formats affected lines to avoid unexpected changes.

Fixes #54888.

Differential Revision: https://reviews.llvm.org/D149643

Noticed in passing. Either way compiles.

Reviewed By: jhuber6

Differential Revision: https://reviews.llvm.org/D149808

To avoid an unnecessary vand.vi

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D149771

This patch changes check for nested TaskGroups so that it allows
parallel execution for TaskGroups. Following pattern would not work
parallelly with current check:

std::function<void()> Fn = [&]() {
  parallel::TaskGroup tg;

  tg.spawn([&]() { });
};

ThreadPool Pool;

Pool.async(Fn);
Pool.async(Fn);

Pool.wait();

One of the TaskGroup would work sequentially as current check
verifies overall number of TaskGroup. Two not nested
TaskGroups can work parallelly but current check prevents this.

Also this patch avoids parallel mode for TaskGroup
in parallel::strategy.ThreadsRequested == 1 case.

This patch is a followup of discussion from D142318

Differential Revision: https://reviews.llvm.org/D148984

KnownBits::ashr now uses the minimum shift amount to try and extend the sign bit

Instead of looking for the existence of a `ret` instruction after disassembly
as a way to check that the right ISA is present in the produced binary, check
instead the ELF header of the file. This is a more reliable solution, since the
alternative can easily result in false positives.

Differential Revision: https://reviews.llvm.org/D149578

Differential Revision: https://reviews.llvm.org/D149823

The step is already expanded in the VPlan. Use this expansion instead.
This is a step towards modeling fixing up IV users in VPlan.

 It also fixes a crash casued by SCEV-expanding the Step expression in
fixupIVUsers, where the IR is in an incomplete state

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D147963

Forward-declare OptionalDiagnostic instead. And turn a few comments in
to doc comments.

in the static functions. Since a Floating is backed by an APFloat, we
don't want to copy that around if it's not necessary.

Add hlfir.forall_mask, hlfir.where, and hlfir.elsewhere operations that
are operations that holds (optionally for hlfir.elsewhere) the
evaluation of a logical mask that controls the evaluation of nested
operations.

They allow representing Fortran forall control mask, as well as where
and eslewhere statements/constructs.

They use the OrderedAssignmentTreeOpInterface since they can all be used
inside Forall and their masks should be fully evaluated for all the
index-value set induced by parent Forall before any of the nested
operations in their body is evaluated.

I initially tried making them into a single operation with some attributes
to make a difference, but I felt this made the verifier/parser/printer and
usages messier/tricky compared to making three distinct operations that
represent the three Fortran feature in a vanilla way.

Differential Revision: https://reviews.llvm.org/D149754