For fixed SVE types, predicates are represented using vectors of i8,
where as for scalable types they are represented using vectors of i1. We
can avoid going through memory for casts between these by bitcasting the
i1 scalable vectors to/from a scalable i8 vector of matching size, which
can then use the existing vector insert/extract logic.

Differential Revision: https://reviews.llvm.org/D106860

Due to an assembler design flaw (IMO), `.symver foo,foo@v1` produces two symbols `foo` and `foo@v1` if `foo` is defined.

* `v1 {};` produces both `foo` and `foo@v1`, but GNU ld only produces `foo@v1`
* `v1 { foo; };` produces both `foo@@v1` and `foo@v1`, but GNU ld only produces `foo@v1`
* `v2 { foo; };` produces both `foo@@v2` and `foo@v1`, matching GNU ld. (Tested by symver.s)

This patch implements the GNU ld behavior by reusing the symbol redirection mechanism
in D92259. The new test symver-non-default.s checks the first two cases.

Without the patch, the second case will produce `foo@v1` and `foo@@v1` which
looks weird and makes foo unnecessarily default versioned.

Note: `.symver foo,foo@v1,remove` exists but the unfortunate `foo` will not go
away anytime soon.

Reviewed By: peter.smith

Differential Revision: https://reviews.llvm.org/D107235

Probably was used for Go at some point...

Depends on D107466.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D107467

It will be needed in more functions like ReportRace
(the plan is to pass it through MemoryAccess to ReportRace)
and this move will allow to split the huge tsan_rtl.h into parts
(e.g. move FastState/Shadow definitions to a separate header).

Depends on D107465.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D107466

Add kAccessExternal memory access flag that denotes
memory accesses with PCs that may have kExternalPCBit set.
In preparation for MemoryAccess refactoring.
Currently unused, but will allow to skip a branch.

Depends on D107464.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D107465

Add kAccessFree memory access flag (similar to kAccessVptr).
In preparation for MemoryAccess refactoring.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D107464

Currently version script patterns are ignored for .symver produced
non-default version (single @) symbols. This makes such symbols
not localizable by `local:`, e.g.

```
.symver foo3_v1,foo3@v1
.globl foo_v1
foo3_v1:

ld.lld --version-script=a.ver -shared a.o
# In a.out, foo3@v1 is incorrectly exported.
```

This patch adds the support:

* Move `config->versionDefinitions[VER_NDX_LOCAL].patterns` to `config->versionDefinitions[versionId].localPatterns`
* Rename `config->versionDefinitions[versionId].patterns` to `config->versionDefinitions[versionId].nonLocalPatterns`
* Allow `findAllByVersion` to find non-default version symbols when `includeNonDefault` is true. (Note: `symtab` keys do not have `@@`)
* Make each pattern check both the unversioned `pat.name` and the versioned `${pat.name}@${v.name}`
* `localPatterns` can localize `${pat.name}@${v.name}`. `nonLocalPatterns` can prevent localization by assigning `verdefIndex` (before `parseSymbolVersion`).

---

If a user notices new `undefined symbol` errors with a version script containing
`local: *;`, the issue is likely due to a missing `global:` pattern.

Reviewed By: peter.smith

Differential Revision: https://reviews.llvm.org/D107234

When loading libomptarget, the init function in libomptarget/src/rtl.cpp
will search for the libomptarget_start_tool function using libdl.
libomptarget_start_tool will pass those OMPT callbacks related to target
constructs to libomptarget

Differential Revision: https://reviews.llvm.org/D99803

GNU ld doesn't support multiple SHF_TLS SHT_NOBITS output sections (it restores
the address after an SHF_TLS SHT_NOBITS section, so consecutive SHF_TLS
SHT_NOBITS sections will have conflicting address ranges).

That said, `threadBssOffset` implements limited support for consecutive SHF_TLS
SHT_NOBITS sections. (SHF_TLS SHT_PROGBITS following a SHF_TLS SHT_NOBITS can still be
incorrect.)

`.` in an output section description of an SHF_TLS SHT_NOBITS section is
incorrect. (https://lists.llvm.org/pipermail/llvm-dev/2021-July/151974.html)

This patch saves the end address of the previous tbss section in
`ctx->tbssAddr`, changes `dot` in the beginning of `assignOffset` so
that `.` evaluation will be correct.

Reviewed By: peter.smith

Differential Revision: https://reviews.llvm.org/D107208

Reviewed By: grosul1, rriddle

Differential Revision: https://reviews.llvm.org/D107472

An insert subvector that is inserting the result of a vector predicate
sized load into undef at index 0, whose result is casted to a predicate
type, can be combined into a direct predicate load. Likewise the same
applies to extract subvector but in reverse.

The purpose of this optimization is to clean up cases that will be
introduced in a later patch where casts to/from predicate types from i8
types will use insert subvector, rather than going through memory early.

This optimization is done in SVEIntrinsicOpts rather than InstCombine to
re-introduce scalable loads as late as possible, to give other
optimizations the best chance possible to do a good job.

Differential Revision: https://reviews.llvm.org/D106549

Making sure the AMX dialect webpage reads better with a short introduction on the purpose of this dialect.

Reviewed By: grosul1, bondhugula

Differential Revision: https://reviews.llvm.org/D107419

This fixes the issue https://bugs.llvm.org/show_bug.cgi?id=51337

Reviewed By: JonChesterfield

Differential Revision: https://reviews.llvm.org/D107468

Don't know how to custom expand this
UNREACHABLE executed at llvm-project/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp:16788

The fix is to provide missing expansions for:
  case ISD::STRICT_FP_TO_UINT:
  case ISD::STRICT_FP_TO_SINT:

A test case is provided.

Reviewed By: dmgreen

Differential Revision: https://reviews.llvm.org/D107452

Zero-width bitfields on AIX pad out to the natral alignment boundary but
do not change the containing records alignment.

Differential Revision: https://reviews.llvm.org/D106900

SCEV-based salvaging in LSR translates SCEVs to DIExpressions. SCEVs may
contain very large integers but the translation does not support
integers greater than 64 bits. This patch adds checks to ensure
conversions of these large integers is not attempted. A regression test
is added to ensure no such translation is attempted.

Reviewed by: StephenTozer

PR: https://bugs.llvm.org/show_bug.cgi?id=51329

Differential Revision: https://reviews.llvm.org/D107438

Differential Revision: https://reviews.llvm.org/D107429

Identified with const-return-type-APInt.

This patch introduces a new code object metadata field, ".kind"
which is used to add support for init and fini kernels.

HSAStreamer will use function attributes, "device-init" and
"device-fini" to distinguish between init and fini kernels from
the regular kernels and will emit metadata with ".kind" set to
"init" and "fini" respectively.

To reduce the number of init and fini kernels, the ctors and
dtors present in the llvm's global.ctors and global.dtors lists
are called from a single init and fini kernel respectively.

Reviewed by: yaxunl

Differential Revision: https://reviews.llvm.org/D105682

Having `NewMask` outside of an if and rebinding `BaseMask` `ArrayRef`
to it is confusing. Instead, just move the `Mask` vector higher up,
and change the code that earlier had no access to it but now does
to use `Mask` instead of `BaseMask`.

This has no other intentional changes.

I want to hoist `Mask` variable higher up,
but then it would clash with this one.
So let's rename this one first.

There are no other intentional changes here other than said rename.

This assert is intended to ensure that the high registers are not
selected when it is passed to one of the thumb UXT instructions. However
it was triggering even for 32 bit where no UXT instruction is emitted.

Fixes PR51313.

Differential Revision: https://reviews.llvm.org/D107363

Given a shuffle mask, if it is picking from an input that is splat
given the current granularity of the shuffle, then adjust the mask
to pick from the same lane of the input as the mask element is in.
This may result in a shuffle being simplified into a blend.

I believe this is correct given that the splat detection matches the one
just above the new code,

My basic thought is that we might be able to get less regressions
by handling multiple insertions of the same value into a vector
if we form broadcasts+blend here, as opposed to D105390,
but i have not really thought this through,
and did not try implementing it yet.

Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D107009

Differential Revision: https://reviews.llvm.org/D107458

This file was previously included transitively via `mlir/Transforms/Passes.h`, but the include has been removed from that file.

Differential Revision: https://reviews.llvm.org/D107455

This attempts to make more of RDA aware of potentially overlapping
subregisters. Some of this was already in place, with it iterating
through MCRegUnitIterators. This also replaces calls to
LiveRegs.contains(..) with !LiveRegs.available(..), and updates the
isValidRegUseOf and isValidRegDefOf to search subregs.

Differential Revision: https://reviews.llvm.org/D107351

There are many downstream users of llvm::dbgs, which is defined in Debug.h. Before D106342, many users included that dependency transitively via the now deleted ViewRegionGraph.h. Adding it back to Transforms/Passes.h for convenience.

Differential Revision: https://reviews.llvm.org/D107451

Match the naming style used by the other 'FeatureFast/FeatureSlow' tuning flags.

Noticed while looking at the feature flag renaming suggested in D107370

Following LLDB tests fail randomly on LLDB Arm/AArch64 Linux buildbots.
We still not have a reliable solution for these tests to pass
consistently. I am marking them skipped for now.

TestBreakpointCallbackCommandSource.py
TestIOHandlerResize.py
TestEditline.py
TestGuiViewLarge.py
TestGuiExpandThreadsTree.py
TestGuiBreakpoints.py

Currently we use passive spinning with internal_sched_yield to wait
in __cxa_guard_acquire/pthread_once. Passive spinning tends to degrade
ungracefully under high load. Use FutexWait/Wake instead.

Depends on D107359.

Reviewed By: vitalybuka, melver

Differential Revision: https://reviews.llvm.org/D107360

The lit tests for `clang-scan-deps` invoke the tool without going through the substitution system. While the test runner correctly picks up the `clang-scan-deps` binary from the build directory, it doesn't print its absolute path. When copying the invocations when reproducing test failures, this can result in `command not found: clang-scan-deps` errors or worse yet: pick up the system `clang-scan-deps`. This patch adds new local `%clang-scan-deps` substitution.

Reviewed By: lxfind, dblaikie

Differential Revision: https://reviews.llvm.org/D107155

Introduce named consts for magic values we use.

Differential Revision: https://reviews.llvm.org/D107445

For some use-cases, it might be useful to be able to turn off modules for C++ in `-cc1`. (The feature is implied by `-std=C++20`.)

This patch exposes the `-fno-cxx-modules` option in `-cc1`.

Reviewed By: arphaman

Differential Revision: https://reviews.llvm.org/D106864

* Add new pass option `print-data-flow-edges`, default value `true`.
* Add new pass option `print-control-flow-edges`, default value `false`.
* Remove `PrintCFGPass`. Same functionality now provided by
  `PrintOpPass`.

Differential Revision: https://reviews.llvm.org/D106342

Currently we effectively duplicate "once" logic for __cxa_guard_acquire
and pthread_once. Unify the implementations.

This is not a no-op change:
 - constants used for pthread_once are changed to match __cxa_guard_acquire
   (__cxa_guard_acquire constants are tied to ABI, but it does not seem
   to be the case for pthread_once)
 - pthread_once now also uses PotentiallyBlockingRegion annotations
 - __cxa_guard_acquire checks thr->in_ignored_lib to skip user synchronization
It's unclear if these 2 differences are intentional or a mere sloppy inconsistency.
Since all tests still pass, let's assume the latter.

Reviewed By: vitalybuka, melver

Differential Revision: https://reviews.llvm.org/D107359

Atomic functions are semi-hot in profiles.
The CHECKs verify values passed by compiler
and they never fired, so replace them with DCHECKs.

Reviewed By: vitalybuka, melver

Differential Revision: https://reviews.llvm.org/D107373

1. Add some comments.
2. Use kInvalidStackID instead of literal 0.
3. Add more LIKELY/UNLIKELY.

Reviewed By: vitalybuka, melver

Differential Revision: https://reviews.llvm.org/D107371