This patch updates several functions in LLVM's IR generation code to accept
an IRBuilder object as an argument, rather than an Instruction that indicates
the insertion point for new instructions.
This change is necessary to handle sophisticated -Ofast optimization cases
from D148558 where it's unclear which instructions should be used as the
insertion point for new operations.

Differential Revision: https://reviews.llvm.org/D148703

They are redundant with the [[fallthrough]]; attribute that follows.

Reviewed By: steakhal

Differential Revision: https://reviews.llvm.org/D151723

Reviewed By: steakhal

Differential Revision: https://reviews.llvm.org/D151726

Add support for the `llvm::TargetExtType` to the MLIR LLVM dialect.

Target extension types were introduced to represent target-specific types, which are opaque to the compiler and optimizations.

The patch also enforces some of the constraints defined for the target extension type in the LLVM language reference manual.

Signed-off-by: Lukas Sommer <lukas.sommer@codeplay.com>

Reviewed By: ftynse, gysit, Dinistro

Differential Revision: https://reviews.llvm.org/D151446

The ability to add attributes to payload IR is useful functionality
independent of any dialect. This is added here through `transform.annotate`
by enabling attributes tied to a `TransformParamTypeInterface` (which
internally refers to an Attribute) to be added to a target operation by
name.

The AnnotateOp does not produce a new handle as no existing handles
should be affected by adding an attribute. Existing attributes on
the payload with the same name will be overwritten.

Differential Revision: https://reviews.llvm.org/D151689

Update collectLoopUniforms to identify uniform pointers using
Legal::isUniform. This is more powerful and  brings pointer
classification here in sync with setCostBasedWideningDecision
which uses isUniformMemOp. The existing mis-match in reasoning
can causes crashes due to D134460, which is fixed by this patch.

Fixes https://github.com/llvm/llvm-project/issues/60831.

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D150991

This allows types to be considered trivially equality comparable if a defaulted hidden friend is used.

Reviewed By: erichkeane

Spies: cfe-commits

Differential Revision: https://reviews.llvm.org/D151623

The transform dialect has been around for a while and is sufficiently
stable at this point. Add the first three chapters of the tutorial
describing its usage and extension.

Reviewed By: springerm

Differential Revision: https://reviews.llvm.org/D151491

This patch uses castAs instead of getAs which will assert if the type
 doesn't match in clang::Type::getSveEltType(clang::ASTContext const &)

Reviewed By: erichkeane

Differential Revision: https://reviews.llvm.org/D151525

This fixes a crash caused by legal treating a scalable GEP as invariant,
but isUniformAfterVectorization does not handle GEPs.

Partially fixes https://github.com/llvm/llvm-project/issues/60831.

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D144434

Depends on: D151111

Differential Revision: https://reviews.llvm.org/D151415

Carries out the lowering of the all intrinsic into HLFIR

Differential Revision: https://reviews.llvm.org/D151111

Adds a new HLFIR operation for the ALL intrinsic according to the
design set out in flang/docs/HighLevel.md

Differential Revision: https://reviews.llvm.org/D151090

This is pretty straight forward in the basic form. I did need to move the slideup matching earlier, but that looks generally profitable on it's own.

As follow ups, I plan to explore the v(f)slide1down variants, and see what I can do to canonicalize the shuffle then insert pattern (see _inverse tests at the end of the vslide1up.ll test).

Differential Revision: https://reviews.llvm.org/D151468

At inserting of InterfaceMap, a debug message lacked a newline, so it repeatedly displayed this message within a single line.
Clean up the debug log by inserting a newline at the end of the message.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D150182

Create SchedWrites, WriteRes for reduction instructions that
are SEW specific. Future patches can use these resources
to customize the behavior of these resources depending on SEW.

Differential Revision: https://reviews.llvm.org/D151470

This fold is buggy if the constant adjustment overflows.
Additionally, since we now canonicalize to min/max intrinsics,
the constants picked here don't actually matter, as long as SPF
still recognizes the pattern.

Fixes https://github.com/llvm/llvm-project/issues/62088.

As raised on Issue #62602 - the IceLake scheduler model is still mainly a copy of the SkylakeServer model. This initial commit just fixes the ROB/MicroOpBufferSize to match the size reported on WikiChip/Agner, further fixes to follow in later commits.

Clarify the restrictions on actions that are implied by the implementation as a post-order walk.

Do not replace uses inside the body of `scf.forall` ops with results of the same op.

Differential Revision: https://reviews.llvm.org/D151706

All result handles must be set in case of success.

Differential Revision: https://reviews.llvm.org/D151705

As reported on https://reviews.llvm.org/D150375#4367861 and
following, this change causes PDT invalidation issues. Revert
it and dependent commits.

This reverts commit 0524534d.
This reverts commit ced90d1f.
This reverts commit 9f992cc9.
This reverts commit 1b123204.

We already early-out at the top of the function.

[X86] X86FixupVectorConstantsPass - attempt to replace full width integer vector constant loads with broadcasts on AVX2+ targets

lowerBuildVectorAsBroadcast will not broadcast splat constants in all cases, resulting in a lot of situations where a full width vector load that has failed to fold but is loading splat constant values could use a broadcast load instruction just as cheaply, and save constant pool space.

This patch adds four new tests for upcoming functionality in LLVM:
* complex-deinterleaving-add-mull-fixed-contract.ll
* complex-deinterleaving-add-mull-scalable-contract.ll
* complex-deinterleaving-add-mull-fixed-fast.ll
* complex-deinterleaving-add-mull-scalable-fast.ll.

These tests were generated from the IR of vectorizable loops, which were
compiled from C++ code using different optimization flags in Clang. Each pair
of tests corresponds to Neon and SVE architectures, respectively, and
each pair contains tests compiled with -Ofast and -O3 -ffp-contract=fast
-ffinite-math-only optimization flags.
The tests were stripped of nnan and ninf flags as they have no impact on the
output.
The primary objective of these tests is to show the various sequences of
complex computations that may be encountered and to demonstrate the ability
of ComplexDeinterleaving to support any ordering.

Depends on D147451

Differential Revision: https://reviews.llvm.org/D148550

This patch simply moves the special handling for `linux` files to a
subdirectory. This is done to make it easier in the future to extend
this support to targets (like the GPU) that will have different
dependencies.

Reviewed By: lntue

Differential Revision: https://reviews.llvm.org/D151231

Cleanup suggested in D150991.

Change __tsan_get_report_loc 6th argument 'size' to unsigned long *

Reviewers: vitalybuka, dvyukov

Differential Revision: https://reviews.llvm.org/D148214

Fixes issue introduced by 0f8e0f42 where SimplifyDemandedBits could crash when trying to extract fp data from broadcasted constants

/data/llvm-project/llvm/lib/Transforms/InstCombine/InstructionCombining.cpp:2467:13: error: unused function 'isMustTailCall' [-Werror,-Wunused-function]
static bool isMustTailCall(Value *V) {
            ^
1 error generated.

We try to fold constant computeKnownBits() with context for return
instructions only. Otherwise, we rely on SimplifyDemandedBits() to
fold instructions with constant known bits.

The presence of this special fold for returns is dangerous, because
it makes our tests lie about what works and what doesn't. Tests are
usually written by returning the result we're interested in, but
will go through this separate code path that is not used for anything
else. This patch removes the special fold.

This primarily regresses patterns of the style "assume(x); return x".
The responsibility of handling such patterns lies with passes like
EarlyCSE/GVN anyway, which will do this reliably, and not just for
returns.

Differential Revision: https://reviews.llvm.org/D151099

This results in improved codegen for half/bf16 libcalls on soft ABIs

Adds a RISCVSubtarget helper method for determining if a soft FP ABI is
being targeted (future bf16 related patches make use of this).

Differential Revision: https://reviews.llvm.org/D151434

This moves memintrinsic interceptors (memcpy/memmove/memset) into a new
file sanitizer_common_interceptors_memintrinsics.inc.

This is in preparation of redefining builtins, however, we must be
careful to not redefine builtins in TUs that define interceptors of the
same name.

In all cases except for MSan, memintrinsic interceptors were moved to a
new TU $tool_interceptors_memintrinsics.cpp. In the case of MSan, it
turns out this is not yet necessary (as shown by the later patch
introducing memcpy tests).

NFC.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D151552

With -ftrivial-auto-var-init, do not emit memset() calls for performance
critical stack variables.

Reviewed By: vitalybuka, dvyukov, MaskRay

Differential Revision: https://reviews.llvm.org/D151551

This reverts commit 421a7f81. Dependency
doesn't seem to be necessary and would pull in all of LLVM's codegen
into mlir users that don't require it.