There is no need for the interface implementations to be exposed, opaque
registration functions are sufficient for all users, similarly to passes.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D97852

Add a Loop Option attribute and generate llvm metadata attached to
branch instructions to control code generation.

Reviewed By: ftynse, mehdi_amini

Differential Revision: https://reviews.llvm.org/D96820

This fixes the following two warnings in code that's only compiled on
arm64:

 - warning: cast from 'const void *' to 'unsigned char *' drops const
   qualifier [-Wcast-qual]
 - warning: embedding a directive within macro arguments has undefined
   behavior [-Wembedded-directive]

This fixes clang warnings (that are treated as errors when running
the test suite):

libcxx/include/string:4409:59: error: definition of dllimport static field [-Werror,-Wdllimport-static-field-def]
               basic_string<_CharT, _Traits, _Allocator>::npos;

The warning is normally not visible as long as the libc++ headers
are treated as system headers.

The same construct is always an error in MSVC.

(One _LIBCPP_FUNC_VIS was added in
2d8f23f5, which broke DLL builds.
59919c4d fixed this by adding another
_LIBCPP_FUNC_VIS on the declaration for consistency, but the underlying
issue remained, that one can't use dllimport here.)

Differential Revision: https://reviews.llvm.org/D97168

Same dangling probes are redundant since they all have the same semantic that is to rely on the counts inference tool to get reasonable count for the same original block. Therefore, there's no need to keep multiple copies of them. I've seen jump threading created tons of redundant dangling probes that slowed down the compiler dramatically. Other optimization passes can also result in redundant probes though without an observed impact so far.

This change removes block-wise redundant dangling probes specifically introduced by jump threading. To support removing redundant dangling probes caused by all other passes, a final function-wise deduplication is also added.

An 18% size win of the .pseudo_probe section was seen for SPEC2017. No performance difference was observed.

Differential Revision: https://reviews.llvm.org/D97482

This change fixes a couple places where the pseudo probe intrinsic blocks optimizations because they are not naturally removable. To unblock those optimizations, the blocking pseudo probes are moved out of the original blocks and tagged dangling, instead of allowing pseudo probes to be literally removed. The reason is that when the original block is removed, we won't be able to sample it. Instead of assigning it a zero weight, moving all its pseudo probes into another block and marking them dangling should allow the counts inference a chance to assign them a more reasonable weight. We have not seen counts quality degradation from our experiments.

The optimizations being unblocked are:

	1. Removing conditional probes for if-converted branches. Conditional probes are tagged dangling when their homing branch arms are folded so that they will not be over-counted.
	2. Unblocking jump threading from removing empty blocks. Pseudo probe prevents jump ...

Dangling probes are the probes associated to an empty block. This usually happens when all real instructions are optimized away from the block. There is a problem with dangling probes during the offline counts processing. The way the sample profiler works is that samples collected on the first physical instruction following a probe will be counted towards the probe. This logically equals to treating the instruction next to a probe as if it is from the same block of the probe. In the dangling probe case, the real instruction following a dangling probe actually starts a new block, and samples collected on the new block may cause issues when counted towards the empty block.

To mitigate this issue, we first try to move around a dangling probe inside its owning block. If there are still native instructions preceding the probe in the same block, we can then use them as a place holder to collect samples for the probe. A pass is added to walk each block backwards looking for probes not followed by any real instruction and moving them before the first real instruction. This is done right before the object emission.

If we are unlucky to find such in-block preceding instructions for a probe, the solution we are taking is to tag such probe as dangling so that the samples reported for them will not be trusted by the compiler. We leave it up to the counts inference algorithm to get such probes a reasonable count. The number `UINT64_MAX` is used to mark sample count as collected for a dangling probe.

Reviewed By: wmi

Differential Revision: https://reviews.llvm.org/D95962

Implements parts of:
    - P0898R3 Standard Library Concepts
    - P1754 Rename concepts to standard_case for C++20, while we still can

Depends on D96660

Reviewed By: ldionne, #libc

Differential Revision: https://reviews.llvm.org/D96742

The `no-aa` pass has been removed with 7b560d40.

Differential Revision: https://reviews.llvm.org/D95416

We often used a sub-optimal dependence class in the past because we
didn't see the argument. Let's make it explicit so we remember to think
about it.

We don't need a bool and an enum to express the three options we
currently have. This makes the interface nicer and much easier to
use optional dependencies. Also avoids mistakes where the bool is
false and enum ignored.

If we know already more than available from a use, we don't need to
invest time on it.

According to ISO/IEC TS 18661-3:2015 _FloatN is interchange floating
point type, extended floating-point type is _FloatNx.

http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2342.pdf

Reviewed By: SjoerdMeijer

Differential revision: https://reviews.llvm.org/D97759

This is an attempt to improve handling of partial overlaps in case of unaligned begin\end.

Existing implementation just bails out if it encounters such cases. Even when it doesn't I believe existing code checking alignment constraints is not quite correct. It tries to ensure alignment of the "later" start/end offset while should be preserving relative alignment between earlier and later start/end.

The idea behind the change is simple. When start/end is not aligned as we wish instead of bailing out let's adjust it as necessary to get desired alignment.

I'll update with performance results as measured by the test-suite...it's still running...

Reviewed By: fhahn

Differential Revision: https://reviews.llvm.org/D93530

Add clspv as a new target for libclc. clspv is an open-source compiler that compiles OpenCL C to Vulkan SPIR-V. Compiles for the spir target.

The clspv target differs from the the spirv target in the following ways:
* fma is modified to use uint2 instead of ulong for mantissas. This results in lower performance fma, but provides a implementation that can be used on more Vulkan devices where 64-bit integer support is less common.
* Use of a software implementation of nextafter because the generic implementation depends on nextafter being a defined builtin function for which clspv has no definition.
* Full optimization of the library (-O3) and no conversion to SPIR-V

This library is close to what would be produced by running opt -O3 < builtins.opt.spirv-mesa3d-.bc > builtins.opt.clspv--.bc and continuing the build from that point.

Reviewer: jvesely

Differential Revision: https://reviews.llvm.org/D94013

This knob is useful for downstream users who want that some of their
libc functions to not be intercepted.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D97740

Implements parts of:
    - P0898R3 Standard Library Concepts
    - P1754 Rename concepts to standard_case for C++20, while we still can

Depends on D96660

Reviewed By: ldionne, #libc

Differential Revision: https://reviews.llvm.org/D96683

statepoint intrinsic can be used in invoke context,
so it should be handled in visitCallBase to cover both call and invoke.

Reviewers: reames, dantrushin
Reviewed By: reames
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D97833

Add option /guard:ehcont

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D96709

Add calls into LanguageRuntime when finding the unwind method to
use out of the 0th (currently executing) stack frame.

Allow for the LanguageRuntimes to indicate if this stack frames
should be treated like a zeroth-frame -- symbolication should be
done based on the saved pc address, not decremented like normal ABI
function calls.

Add methods to RegisterContext and StackFrame to get a pc value
suitable for symbolication, to reduce the number of places in lldb
where we decrement the saved pc values before symbolication.

<rdar://problem/70398009>
Differential Revision: https://reviews.llvm.org/D97644

This is just a shorter synonym for `__identity<T>::type`.
Use it consistently throughout, where possible.

There is still some metaprogramming in <memory> and <variant>
where `__identity` is being used _without_ immediately calling
`::type` on it; but this is the unusual case, and it will become
even less usual as we start deliberately protecting certain types
against deduction (e.g. D97742).

Differential Revision: https://reviews.llvm.org/D97862

Implements parts of:
    - P0898R3 Standard Library Concepts
    - P1754 Rename concepts to standard_case for C++20, while we still can

Depends on D96657

Reviewed By: ldionne, Mordante, #libc

Differential Revision: https://reviews.llvm.org/D96660

Found with exhaustive testing, it is possible that a while loop
appears in between chainable for loops. As long as we don't
scalarize reductions in while loops, this means we need to
terminate the chain at the while. This also refactors the
reduction code into more readable helper methods.

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D97886

This patch is an update to LangRef by describing lifetime intrinsics' behavior
by following the description of MIR's LIFETIME_START/LIFETIME_END markers
at StackColoring.cpp (https://github.com/llvm/llvm-project/blob/eb44682d671d66e422b02595a636050582a4d84a/llvm/lib/CodeGen/StackColoring.cpp#L163) and the discussion in llvm-dev.

In order to explicitly define the meaning of an object lifetime, I added 'Object Lifetime' subsection.

Reviewed By: nlopes

Differential Revision: https://reviews.llvm.org/D94002

The support for attributes closely maps that of Types (basically 1-1) given that Attributes are defined in exactly the same way as Types. All of the current ODS TypeDef classes get an Attr equivalent. The generation of the attribute classes themselves share the same generator as types.

Differential Revision: https://reviews.llvm.org/D97589

[RISCV] Make use of the required features in BuiltinInfo to store that V extension builtins require 'experimental-v'.

Use that to print the diagnostic in SemaChecking instead of
listing all of the builtins in a switch.

With the required features, IR generation will also be able
to error on this. Checking this here allows us to have a RISCV
focused error message.

Reviewed By: HsiangKai

Differential Revision: https://reviews.llvm.org/D97826

IR symbol table does not parse inline asm. A symbol only referenced by inline
asm is not in the IR symbol table, so LTO does not know that the definition (in
another translation unit) is referenced and may internalize it, even if that
definition has `__attribute__((used))` (which lowers to `llvm.compiler.used` on
ELF targets since D97446).

```
// cabac.c
__attribute__((used)) const uint8_t ff_h264_cabac_tables[...] = {...};

// h264_cabac.c
  asm("lea ff_h264_cabac_tables(%rip), %0" : ...);
```

`__attribute__((used))` is the recommended way to tell the compiler there may
be inline asm references, so the usage is perfectly fine. This patch
conservatively sets the `FB_used` bit on `llvm.compiler.used` symbols to work
around the IR symbol table limitation. Note: before D97446, Clang never emitted
symbols in the `llvm.compiler.used` list, so this change does not punish any
Clang emitted global object.

Without the patch, `ff_h264_cabac_tables` may be assi...

On AIX, the default arch level should match the minimum supported arch level of the OS version.

Differential Revision: https://reviews.llvm.org/D97823

This better matches the actual IR concept that is being modeled, and is consistent with how the rest of PDL is structured.

Differential Revision: https://reviews.llvm.org/D95718

This type represents a range of positional values. It will be used in followup revisions to add support for variadic constructs to PDL, such as operand and result ranges.

Differential Revision: https://reviews.llvm.org/D95717

See pr46990(https://bugs.llvm.org/show_bug.cgi?id=46990). LICM should not sink store instructions to loop exit blocks which cross coro.suspend intrinsics. This breaks semantic of coro.suspend intrinsic which return to caller directly. Also this leads to use-after-free if the coroutine is freed before control returns to the caller in multithread environment.

This patch disable promotion by check whether loop contains coro.suspend intrinsics.
This is a resubmit of D86190.
Disabling LICM for loops with coroutine suspension is a better option not only for correctness purpose but also for performance purpose.
In most cases LICM sinks memory operations. In the case of coroutine, sinking memory operation out of the loop does not improve performance since coroutien needs to get data from the frame anyway. In fact LICM would hurt coroutine performance since it adds more entries to the frame.

Differential Revision: https://reviews.llvm.org/D96928

`__llvm_prf_nm` is compressed if zlib is available. In addition, its size may not be that stable.

The compiler needs to mark register $x0 as live in for the following case.

    $x1 = ADDXri $sp, 16, 0
    BL @spam, csr_darwin_aarch64_aapcs, implicit-def dead $lr, implicit $sp, implicit $x0, implicit killed $x1, implicit-def $sp, implicit-def dead $x0

Reviewed By: paquette

Differential Revision: https://reviews.llvm.org/D95267