This patch adds support for producer-consumer fusion scenarios with
multiple producer stores to the AffineLoopFusion pass. The patch
introduces some changes to the producer-consumer algorithm, including:

* For a given consumer loop, producer-consumer fusion iterates over its
producer candidates until a fixed point is reached.

* Producer candidates are gathered beforehand for each iteration of the
consumer loop and visited in reverse program order (not strictly guaranteed)
to maximize the number of loops fused per iteration.

In general, these changes were needed to simplify the multi-store producer
support and remove some of the workarounds that were introduced in the past
to support more fusion cases under the single-store producer limitation.

This patch also preserves the existing functionality of AffineLoopFusion with
one minor change in behavior. Producer-consumer fusion didn't fuse scenarios
with escaping memrefs and multiple outgoing edges (from a single store).
Multi-store producer scenarios will usually (always?) have multiple outgoing
edges so we couldn't fuse any with escaping memrefs, which would greatly limit
the applicability of this new feature. Therefore, the patch enables fusion for
these scenarios. Please, see modified tests for specific details.

Reviewed By: andydavis1, bondhugula

Differential Revision: https://reviews.llvm.org/D92876

Replaced CUDA builtin vars with LLVM intrinsics such that we don't need
definitions of those intrinsics.

Reviewed By: JonChesterfield

Differential Revision: https://reviews.llvm.org/D95013

The loop-unswitch transform should not be performed on a loop whose
condition is divergent. For this to happen correctly, divergence
analysis must be available. The existing divergence analysis has not
been ported to the new pass manager yet. As a result, loop unswitching
on the new pass manager is currently unsafe on targets that care about
divergence.

This test is temporarily disabled to unblock work on the new pass
manager. The issue is now tracked in bug 48819.

Reviewed By: foad

Differential Revision: https://reviews.llvm.org/D95051

This is another step towards removing `OperationData` and
fixing FMF matching/propagation bugs when forming reductions.

We were able to remove almost all of the state from
OperationData, so these don't make sense as members
of that class - just pass the RecurKind in as a param.

More streamlining is possible, but I'm trying to avoid
logic/typo bugs while fixing this. Eventually, we should
not need the `OperationData` class.

We were able to remove almost all of the state from
OperationData, so these don't make sense as members
of that class - just pass the RecurKind in as a param.

The documentation for contributing to LLVM currently links to the section
explaining how to submit a Phabricator review using the web interface.
I believe it would be better to link to the general page for using
Phabricator instead, which explains how to sign up with Phabricator,
and also how to submit patches using either the web interface or the
command-line.

I think this is worth changing because what currently *appears* to be our
preferred way of submitting a patch (through the web interface) isn't
actually what we prefer. Indeed, patches submitted from the command-line
have more meta-data available (such as which repository the patch targets),
and also can't suffer from missing context.

Differential Revision: https://reviews.llvm.org/D94929

Loop peeling assumes that the loop's latch is a conditional branch.  Add
a check to canPeel that explicitly checks for this, and testcases that
otherwise fail an assertion when trying to peel a loop whose back-edge
is a switch case or the non-unwind edge of an invoke.

Reviewed By: skatkov, fhahn

Differential Revision: https://reviews.llvm.org/D94995

This reduces template bloat, but more importantly, makes it possible to
construct one from clang-query without template types.

Differential Revision: https://reviews.llvm.org/D94879

[libomptarget][devicertl] Wrap source in declare target pragmas

Factored out of D93135 / D94745. C++ and cuda ignore unknown pragmas
so this is a NFC for the current implementation language. Removes noise
from patches for building deviceRTL as openmp.

Reviewed By: tianshilei1992

Differential Revision: https://reviews.llvm.org/D95048

Currently all these tests are XFAILED on Linux even though the problem
only seems to be with the few checks that look at collation. To retain
test coverage this splits the locale-dependent tests into a separate
.pass.cpp that is XFAILed as before.
This commit also XFAILs the locale-dependent tests on FreeBSD since the
[=M=] and [.ch.] behaviour for cs_CZ also doesn't seem to match the
behaviour that is expected by these tests.

Reviewed By: #libc, ldionne

Differential Revision: https://reviews.llvm.org/D94969

Differential Revision: https://reviews.llvm.org/D94878

Add DemandedElts support inside the TRUNCATE analysis.

Differential Revision: https://reviews.llvm.org/D56387

Combined with 'da986511 - Revert "DR2064:
decltype(E) is only a dependent', this change (5a391d38) caused verifier
errors when building Chromium. See https://crbug.com/1168494#c1 for a
reproducer.

Additionally it reverts changes that were dependent on this one, see
below.

> Following up on PR48517, fix handling of template arguments that refer
> to dependent declarations.
>
> Treat an id-expression that names a local variable in a templated
> function as being instantiation-dependent.
>
> This addresses a language defect whereby a reference to a dependent
> declaration can be formed without any construct being value-dependent.
> Fixing that through value-dependence turns out to be problematic, so
> instead this patch takes the approach (proposed on the core reflector)
> of allowing the use of pointers or references to (but not values of)
> dependent declarations inside value-dependent expressions, and instead
> treating template arguments as dependent if they evaluate to a constant
> involving such dependent declarations.
>
> This ends up affecting a bunch of OpenMP tests, due to OpenMP
> imprecisely handling instantiation-dependent constructs, bailing out
> early instead of processing dependent constructs to the extent possible
> when handling the template.
>
> Previously committed as 8c1f2d15, and
> reverted because a dependency commit was reverted.

This reverts commit 5a391d38.

It also restores clang/test/SemaCXX/coroutines.cpp to its state before
da986511.

Revert "[c++20] P1907R1: Support for generalized non-type template arguments of scalar type."

> Previously committed as 9e08e51a, and
> reverted because a dependency commit was reverted. This incorporates the
> following follow-on commits that were also reverted:
>
> 7e84aa1b by Simon Pilgrim
> ed13d8c6 by me
> 95c7b6ca by Sam McCall
> 430d5d84 by Dave Zarzycki

This reverts commit 4b574008.

Revert "[msabi] Mangle a template argument referring to array-to-pointer decay"

> [msabi] Mangle a template argument referring to array-to-pointer decay
> applied to an array the same as the array itself.
>
> This follows MS ABI, and corrects a regression from the implementation
> of generalized non-type template parameters, where we "forgot" how to
> mangle this case.

This reverts commit 18e093fa.

This reverts commit c056f824.

That commit causes build failures.

As discussed on D56387, if we're shifting to extract the upper/lower half of a vXi64 vector then we're actually better off performing this at the subvector level as its very likely to fold into something.

combineConcatVectorOps can perform this in reverse if necessary.

Differential Revision: https://reviews.llvm.org/D94822

Add the aarch64[_be]-*-gnu_ilp32 targets to support the GNU ILP32 ABI for AArch64.

The needed codegen changes were mostly already implemented in D61259, which added support for the watchOS ILP32 ABI. The main changes are:
- Wiring up the new target to enable ILP32 codegen and MC.
- ILP32 va_list support.
- ILP32 TLSDESC relocation support.

There was existing MC support for ELF ILP32 relocations from D25159 which could be enabled by passing "-target-abi ilp32" to llvm-mc. This was changed to check for "gnu_ilp32" in the target triple instead. This shouldn't cause any issues since the existing support was slightly broken: it was generating ELF64 objects instead of the ELF32 object files expected by the GNU ILP32 toolchain.

This target has been tested by running the full rustc testsuite on a big-endian ILP32 system based on the GCC ILP32 toolchain.

Reviewed By: kristof.beyls

Differential Revision: https://reviews.llvm.org/D94143

This patch pre-commits a test case with wrong exit count
analysis for D92367.

Reviewed by: mkazantsev

Differential Revision: https://reviews.llvm.org/D94657

The pass analysis uses "sets" implemented using a SmallVector type
to keep track of Used, Preserved, Required and RequiredTransitive
passes. When having nested analyses we could end up with duplicates
in those sets, as there was no checks to see if a pass already
existed in the "set" before pushing to the vectors. This idea with
this patch is to avoid such duplicates by avoiding pushing elements
that already is contained when adding elements to those sets.

To align with the above PMDataManager::collectRequiredAndUsedAnalyses
is changed to skip adding both the Required and RequiredTransitive
passes to its result vectors (since RequiredTransitive always is
a subset of Required we ended up with duplicates when traversing
both sets).

Main goal with this is to avoid spending time verifying the same
analysis mulitple times in PMDataManager::verifyPreservedAnalysis
when iterating over the Preserved "set". It is assumed that removing
duplicates from a "set" shouldn't have any other negative impact
(I have not seen any problems so far). If this ends up causing
problems one could do some uniqueness filtering of the vector being
traversed in verifyPreservedAnalysis instead.

Reviewed By: foad

Differential Revision: https://reviews.llvm.org/D94416

find-references on `virtual void meth^od() = 0` will include override references.

Differential Revision: https://reviews.llvm.org/D94390

Depends on D94970

Differential Revision: https://reviews.llvm.org/D94971

Differential Revision: https://reviews.llvm.org/D94970

NameMatch could be a float close to zero, in such cases we were
dividing by zero and moreover propogating a "NaN" to clients, which is invalid
per JSON.

This fixes the issue by only using Quality scores whenever the NameMatch is low,
as we do in CodeCompletion ranking.

Fixes https://github.com/clangd/clangd/issues/648.

Differential Revision: https://reviews.llvm.org/D94755

This puts it in alphabetical order, matching the rest of the list.

Reviewed by: MaskRay, saugustine

Differential Revision: https://reviews.llvm.org/D94481

This adds some test coverage for
caafdf07, which relaxed an assertion
to only require a unique exit block.

In 8031785f the temporary object being built was moved to %t/main.o,
but not all run lines were updated to reflect this. Observe the failure
on this buildbot:

  http://lab.llvm.org:8011/#/builders/5/builds/3646/steps/9/logs/stdio

It might pass locally for some people due to a stale %t.o hanging around
the build directory.

Add a check if regions do not implement the RegionBranchOpInterface. This is not
allowed in the current deallocation steps. Furthermore, we handle edge-cases,
where a single region is attached and the parent operation has no results.

This fixes: https://bugs.llvm.org/show_bug.cgi?id=48575

Differential Revision: https://reviews.llvm.org/D94586

The runtime-wrappers depend on LLVMSupport, pulling in static initialization code (e.g. command line arguments). Dynamically loading multiple such libraries results in ODR violoations.

So far this has not been an issue, but in D94421, I would like to load both the async-runtime and the cuda-runtime-wrappers as part of a cuda-runner integration test. When doing this, code that asserts that an option category is only registered once fails (note that I've only experienced this in Google's bazel where the async-runtime depends on LLVMSupport, but a similar issue would happen in cmake if more than one runtime-wrapper starts to depend on LLVMSupport).

The underlying issue is that we have a mix of static and dynamic linking. If all dependencies were loaded as shared objects (i.e. if LLVMSupport was linked dynamically to the runtime wrappers), each dependency would only get loaded once. However, linking dependencies dynamically would require special attention to paths (one could dynamically load the dependencies first given explicit paths). The simpler approach seems to be to link all dependencies statically into a single shared object.

This change basically applies the same logic that we have in the c_runner_utils: we have a shared object target that can be loaded dynamically, and we have a static library target that can be linked to other runtime-wrapper shared object targets.

Reviewed By: herhut

Differential Revision: https://reviews.llvm.org/D94399

Reviewed by xbolva00

Reviewers: llvm-commits, MatzeB, craig.topper, kparzysz, efriedma, pengfei, wxiao3, xbolva00

Subscribers: llvm-commits, xbolva00

Differential Revision: https://reviews.llvm.org/D94904

Summary: This is to address bug48712.
The solution in this patch is that when we want to merge two variable a
into the storage frame of variable b only if the alignment of a is
multiple of b.
There may be other strategies. But now I think they are hard to handle
and benefit little. Or we can implement them in the future.

Test-plan: check-llvm

Reviewers: jmorse, lxfind, junparser

Differential Revision: https://reviews.llvm.org/D94891

If constants are hidden behind G_ANYEXT we can treat them same way as G_SEXT.
For that purpose we extend getConstantVRegValWithLookThrough with option
to handle G_ANYEXT same way as G_SEXT.

Differential Revision: https://reviews.llvm.org/D92219

And mark a couple to be retired afther the next release branch.

Differential Revision: https://reviews.llvm.org/D94727

These force a couple of flags or that are now on by default.
So the flags don't currently do anything unless the compile command has
-fno-recovery-ast explicitly.

(For turning recovery *off* for debugging we can inject the flag with config)

This leaves the command-line flags around with no effect, I'm planning to add
a "retired flag" mechanism shortly in a separate patch.

Differential Revision: https://reviews.llvm.org/D94724

This is on the critical path (it blocks getting the compile command for
the first file).

It's not trivially fast: it involves processing all filenames in the CDB
and doing some IO to look for shadowing CDBs.

And we may make this slower soon - making CDB configurable implies evaluating
the config for each listed to see which ones really are owned by the
broadcasted CDB.

Differential Revision: https://reviews.llvm.org/D94606

In preparation for moving DirectoryBasedCompilationDatabase broadcasting off
the main thread.

Differential Revision: https://reviews.llvm.org/D94603

D94904 Reviewed by xbolva00

Reviewers for D94904: llvm-commits, MatzeB, craig.topper, kparzysz, efriedma, pengfei, wxiao3, xbolva00

Subscribers for D94904: llvm-commits, xbolva00

D94904 Differential Revision: https://reviews.llvm.org/D94904

This patch adds marshalling information to more options.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D94957

With tfe on there can be a vgpr write to vdata+1.
Add tablegen support for 5 register vdata store.
This is required for 4 register vdata store with tfe.

Differential Revision: https://reviews.llvm.org/D94960