[PowerPC][lld] Account for additional X-Forms -> D-Form/DS-Forms load/stores when relaxing initial-exec to local-exec

D153645 added additional X-Form load/stores that can be generated for TLS accesses.
However, these added instructions have not been accounted for in lld. As a result,
lld does not know how to handle them and cannot relax initial-exec to local-exec
when the initial-exec sequence contains these additional load/stores.

This patch aims to resolve https://github.com/llvm/llvm-project/issues/64424.

Differential Revision: https://reviews.llvm.org/D158197

This function implements the `abort` function on the GPU. The
implementation here closely mirros the `exit` call where we first
synchornize with the RPC server to make sure it's listening and then we
exit on the GPU.

I was unsure if this should be a simple `__builtin_assert` on the GPU. I
elected to go with an RPC approach to make this a more "true" `abort`
call. That is, it should invoke some signal handlers and exit with the
proper code according to the implemented C library on the server.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D159210

This simplifies the code and unifies code-paths to use a single function
for emitting streaming-mode changes.

The inbox/outbox loads are performed by the current warp, not a single thread.

The outbox load indicates whether a port has been successfully opened. If some
lanes in the warp think it has and others think the port open failed, as the
warp happened to be diverged when the load occurred, all the subsequent control
flow will be incorrect.

The inbox load indicates whether the machine on the other side of the RPC channel
has progressed. If lanes in the warp have different ideas about that, some will
try to progress their state transition while others won't. As far as the RPC layer
is concerned this is a performance problem and not a correctness one - none of the lanes
can start the transition early, only miss it and start late - but in practice the calls
layered on top of RPC do not have the interface required to detect this event and retry
the load on the stalled lanes, so the calls layered on top will be broken.

None of this is broken on amdgpu, but it's likely that the readfirstlane will have
beneficial performance properties there. Possible significant enough that it's
worth landing this ahead of fixing gpu::broadcast_value on volta.

Essentially volta wasn't adequately considered when writing this part of the protocol.
It's a bug present in the initial prototype and propagated thus far, because none of
the test cases push volta into a warp diverged state in the middle of the RPC sequence.

We should have some test cases for volta where port_open and equivalent are called
from diverged warps.

Reviewed By: jhuber6

Differential Revision: https://reviews.llvm.org/D159276

We noticed this same issue in our own implementation of abs_diff, and
the same issue also came up in the abs_diff reference function in the
OpenCL CTS.

Reviewed By: rjodinchr

Differential Revision: https://reviews.llvm.org/D159275

This reverts commit de0df639.

It was reverted due to regression in HIP unit test on Windows:

 In file included from C:\hip-tests\catch\unit\graph\hipGraphClone.cc:37:

 In file included from C:\hip-tests\catch\.\include\hip_test_common.hh:24:

 In file included from C:\hip-tests\catch\.\include/hip_test_context.hh:24:

 In file included from C:/install/native/Release/x64/hip/include\hip/hip_runtime.h:54:

 C:/dk/win\vc\14.31.31107\include\thread:76:70: error: cannot initialize a parameter of type '_beginthreadex_proc_type' (aka 'unsigned int (*)(void *) __attribute__((stdcall))') with an lvalue of type 'const unsigned int (*)(void *) noexcept __attribute__((stdcall))': different exception specifications

    76 |             reinterpret_cast<void*>(_CSTD _beginthreadex(nullptr, 0, _Invoker_proc, _Decay_copied.get(), 0, &_Thr._Id));

       |                                                                      ^~~~~~~~~~~~~

 C:\hip-tests\catch\unit\graph\hipGraphClone.cc:290:21) &>' requested here

    90 |         _Start(_STD forward<_Fn>(_Fx), _STD forward<_Args>(_Ax)...);

       |         ^

 C:\hip-tests\catch\unit\graph\hipGraphClone.cc:290:21) &, 0>' requested here

   311 |     std::thread t(lambdaFunc);

       |                 ^

 C:/dk/win\ms_wdk\e22621\Include\10.0.22621.0\ucrt\process.h:99:40: note: passing argument to parameter '_StartAddress' here

    99 |     _In_      _beginthreadex_proc_type _StartAddress,

       |                                        ^

 1 error generated when compiling for gfx1030.

There were 3 definitions of the mergeDefaultFunctionDefinitionAttributes
function: A private implementation, a version exposed in CodeGen, a
version exposed in CodeGenModule.

This patch removes the private and the CodeGenModule versions and keeps
a single definition in CodeGen.

Reviewed By: jhuber6

Differential Revision: https://reviews.llvm.org/D159256

Allow specialization of functions with "dynamic" denormal modes to a
known IEEE or DAZ mode based on callers. This should make it possible
to implement a is-denormal-flushing-enabled test using
llvm.canonicalize and have it be free after LTO.

https://reviews.llvm.org/D156129

[X86] combineCMP - attempt to simplify KSHIFTR mask element extractions when just comparing against zero

We can just bitcast the pre-shifted mask as an integer and use TEST/BT directly.

This can be extended further to better handle sub-i8 mask cases, but just getting rid of KSHIFTR nodes makes a notable difference.

With the R_Group options, invalid values e.g. '-Rpa' will not emit
a warning like clang. This patch enables warning reporting, as
well as suggestions on what option the user intended to select.

Depends on D158174 and D158436. The former, adds backend
support to R_Group options while the latter, implements
regex support with some tests refactoring that cause a merge
conflict.

Reviewed By: awarzynski

Differential Revision: https://reviews.llvm.org/D158593

Directly traverse `ConceptReference`s in FindTarget.cpp.

There is no need for the extra logic for `AutoTypeLoc`s in SemanticHightlighting.cpp as the concept information is stored in a `ConceptReference` which is now traversed.

Differential Revision: https://reviews.llvm.org/D159268

Functions are always callable operations and thus every operation
implementing the `FunctionOpInterface` also implements the
`CallableOpInterface`. The only exception was the FuncOp in the toy
example. To make implementation of the `FunctionOpInterface` easier,
this commit lets `FunctionOpInterface` inherit from
`CallableOpInterface` and merges some of their methods. More precisely,
the `CallableOpInterface` has methods to get the argument and result
attributes and a method to get the result types of the callable region.
These methods are always implemented the same way as their analogues in
`FunctionOpInterface` and thus this commit moves all the argument and
result attribute handling methods to the callable interface as well as
the methods to get the argument and result types. The
`FuntionOpInterface` then does not have to declare them as well, but
just inherits them from the `CallableOpInterface`.
Adding the inheritance relation also required to move the
`FunctionOpInterface` from the IR directory to the Interfaces directory
since IR should not depend on Interfaces.

Reviewed By: jpienaar, springerm

Differential Revision: https://reviews.llvm.org/D157988

This has always worked but had no coverage. Adding testing now so that
later I can refactor the save/restore code safely.

Reviewed By: omjavaid

Differential Revision: https://reviews.llvm.org/D157488

As suggested by D159198

The MatchTable-based GlobalISel Combiner backend is the new default. There are no in-tree users left of the old backend.

- Removed implementation of old MatchDAG-based Combiner, including tests, the backend itself and all supporting code.
- Renamed MatchTable backend to `GlobalISelCombinerEmitter.cpp` + removed "-matchtable" from its CL option.
    - no need to have a verbose name as it's the only backend left now.

Reviewed By: aemerson

Differential Revision: https://reviews.llvm.org/D158710

Add regex handling for all variations of OPT_R_Joined, i.e.
`-Rpass`, `-Rpass-analysis`, `-Rpass-missed`.

Depends on D158174. That patch implements backend support for
R_Group options.

Reviewed By: awarzynski

Differential Revision: https://reviews.llvm.org/D158436

This revision adds support for unstructured control flow to the bufferization infrastructure. In particular: regions with multiple blocks, `cf.br`, `cf.cond_br`.

Two helper templates are added to `BufferizableOpInterface.h`, which can be implemented by ops that supported unstructured control flow in their regions (e.g., `func.func`) and ops that branch to another block (e.g., `cf.br`).

A block signature is always bufferized together with the op that owns the block.

Differential Revision: https://reviews.llvm.org/D158094

When replacing ComplexDeinterleavingPass::ReductionOperation, we can do it
either from the Real or Imaginary part. The correct way is to take whichever
is later in the BasicBlock, but before the patch, we just always took the
Real part.

Fixes https://github.com/llvm/llvm-project/issues/65044

Differential Revision: https://reviews.llvm.org/D159209

AArch64 incorrectly nests ADJCALLSTACKDOWN/ADJCALLSTACKUP which fails
to verify with expensive checks enabled.

See https://github.com/llvm/llvm-project/issues/62137 and
https://github.com/llvm/llvm-project/issues/62138.

Differential Revision: https://reviews.llvm.org/D157630

While doing some refactoring I forgot to carry over the copying in of
SIMD data in normal mode, but no tests failed.

Turns out, it's very easy for us to get the restore wrong because
even if you forget the memcopy, setting the buffer to valid may
just read the data you had before the expression evaluation.

So I've extended the SVE SIMD testing (which includes the plain SIMD mode)
to check expression save/restore. This is the only test that fails
if you forget to do `m_fpu_is_valid = true` so I take from that, that
prior to this it wasn't tested at all.

As a bonus, we now have coverage of the same thing for SVE and SSVE modes.

Reviewed By: omjavaid

Differential Revision: https://reviews.llvm.org/D157000

Previously we would "process continue" then wait for the number of
threads to be 3 before proceeding with the test.

Testing this on QEMU I saw it would sometimes get stuck at this check,
with one of the threads on a breakpoint before the other had started.
We do want it to be on a breakpoint, but we need the other thread to have
at least started so lldb can interact with both.

I've also seen it timeout on the Graviton buildbot, likely the same
cause.

To fix this add 2 variables to stall either thread until the other
has started up. Then it doesn't matter which one hits its breakpoint
first, the test will just continue the one that didn't, until both
are on the expected breakpoint.

Differential Revision: https://reviews.llvm.org/D157967

This patch adds a concept AST node (`ConceptLoc`) and uses it at the corresponding places.

There are three objects that might have constraints via concepts:
`TypeConstraint`,  `ConceptSpecializationExpr` and `AutoTypeLoc`.
The first two inherit from `ConceptReference` while the latter has
the information about a possible constraint directly stored in `AutoTypeLocInfo`. It would be nice if the concept information would be stored the same way in all three cases.

Moreover the current structure makes it difficult to deal with these concepts. For example in Clangd accessing the locations of constraints of a `AutoTypeLoc` can only be done with quite ugly hacks.

So we think that it makes sense to create a new AST node for such concepts.

In details we propose the following:
- Rename `ConceptReference` to `ConceptLoc` (or something else what is approriate)
and make it the new AST node.
- `TypeConstraint` and `ConceptSpecializationExpr` do not longer inherit from `ConceptReference` but store a pointer to a `ConceptLoc`.
- `AutoTypeLoc` stores a pointer to `ConceptLoc` instead of storing the concept info in `AutoTypeLocInfo`.

This patch implements a first version of this idea which compiles and where the existing tests pass.
To make this patch as small as possible we keep the existing member functions to access concept data. Later these can be replaced by directly calling the corresponding functions of the `ConceptLoc`s.

Differential Revision: https://reviews.llvm.org/D155858

When compiling the builtins with the undefined behavior sanitizer and running
testcases you end up with the following warning:

UBSan: fp_fixint_impl.inc:39:42: left shift of 8388608 by 40 places cannot be represented in type 'fixint_t' (aka 'long long')
UBSan: fp_fixint_impl.inc:39:17: signed integer overflow: -1 * -9223372036854775808 cannot be represented in type 'fixint_t' (aka 'long long')

This can be avoided by doing the shift and the multiplication in a matching
unsigned variant of the type.

The added test only trigger the intended signed overflow case when the builtins
are built with -D__SOFTFP__.

This was found in an out of tree target.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D159069

The main function of the LSP server needs to load the dialects and
similar that the server should be able to understand. When extensions
where introduced, the loading of the extensions was apparently not added
to its main functions, so ops from extensions were previously not
recognized by the server. This patch registers all extensions through
the existing convenience function, and also registers the
TestDynDialect, which `mlir-opt`s main function also registers.

Reviewed By: springerm

Differential Revision: https://reviews.llvm.org/D159091

For inline asm with memory operands, we can merge the offset into
the second operand of memory constraint operands.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D158062

Tests for callbr, multi-operands and multi-asm are added.

Reviewed By: wangpc, craig.topper

Differential Revision: https://reviews.llvm.org/D158149

Otherwise they fail in builds configured with
-DCLANG_ENABLE_STATIC_ANALYZER=OFF. Follow-up to
3c9988f8.

This revision supports --print-supported-extensions,
it prints out all of the extensions and corresponding version supported.

Reviewed By: craig.topper, kito-cheng

Differential Revision: https://reviews.llvm.org/D146054

Moves the lowering of `bufferization.dealloc` to memref into a separate pass,
but still registers the pattern in the conversion pass.  This is helpful when
some tensor values (and thus `to_memref` or `to_tensor` operations) still
remain, e.g., when the function boundaries are not converted, or when constant
tensors are converted to memref.get_global at a later point.

However, it is still recommended to perform all bufferization before
deallocation to avoid memory leaks as all memref allocations inserted after the
deallocation pass was applied, have to be handled manually.

Note: The buffer deallocation pass assumes that memref values defined by
`bufferization.to_memref` don't return ownership and don't have to be
deallocated. `bufferization.to_tensor` operations are handled similarly to
`bufferization.clone` operations with the exception that the result value is
not handled because it's a tensor (not a memref).

Reviewed By: springerm

Differential Revision: https://reviews.llvm.org/D159180

This trait is needed so that unstructured control flow is not inlined into "scf.while" ops.

Note: The two regions of "scf.while" are already defined as `SizedRegion<1>`. `SingleBlock` can be queried from C++, `SizedRegion<n>` not.

Fixes #64976.

Differential Revision: https://reviews.llvm.org/D159199

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D159245

definition to support operators with multiple outputs.

Differential Revision: https://reviews.llvm.org/D152388

We only support Clang on windows, so this code path is never taken.

Reviewed By: #libc, Mordante

Spies: Mordante, libcxx-commits

Differential Revision: https://reviews.llvm.org/D158230

Differential Revision: https://reviews.llvm.org/D158257