Compiler can generate DIE References that are invalid. Previously BOLT could
assert when writing out IR to .debug_info. Changed where DIE offsets are changed
so that it's always done. Thus making sure that assert is not triggered.

Added more specific warnings, and ability to print out invalid referenced DIE
offset when verbosity >=1.

Reviewed By: Amir

Differential Revision: https://reviews.llvm.org/D157746

This bug crept in when CU partitioning was introduced. It manifests itself when
there are CUs that use location lists that come before CUs that are part of
thin-lto. BOLT processes CUs with cross CU references first (these are produced
by thin-lto). When we wrote out all the location lists we did it in original
order. Since DWARF4 uses offsets directly in to .debug_loc those offsets in DIEs
became wrong.

Reviewed By: Amir

Differential Revision: https://reviews.llvm.org/D157908

Reviewed By: #libc, #libc_abi, Mordante

Spies: arphaman, Mordante, libcxx-commits, arichardson

Differential Revision: https://reviews.llvm.org/D157060

[libc++][PSTL] Simplify the partitioning algorithm until we have better data to know how to chunk better

The current chunking strategy is very bad for sorting, and we don't really know how to chunk in general. This fixes the performance problem for sorting.

Reviewed By: ldionne, #libc

Spies: libcxx-commits, krytarowski

Differential Revision: https://reviews.llvm.org/D155531

When LLVM is build with `LLVM_ENABLE_EXPENSIVE_CHECKS=ON` option the
following C code snippet:

    struct t {
      int a;
    } __attribute__((preserve_access_index));

    void test(struct t *t) {
      t->a = 42;
    }

Causes an assertion:

$ clang -g -O2 -c --target=bpf -mcpu=v2 t.c -o /dev/null

Function Live Ins: $r1 in %0

bb.0.entry:
  liveins: $r1
  DBG_VALUE $r1, $noreg, !"t", ...
  %0:gpr = COPY $r1
  DBG_VALUE %0:gpr, $noreg, !"t", ...
  %1:gpr = LD_imm64 @"llvm.t:0:0$0:0"
  %3:gpr = ADD_rr %0:gpr(tied-def 0), killed %1:gpr
  %4:gpr = MOV_ri 42
  CORE_MEM killed %4:gpr, 411, %0:gpr, @"llvm.t:0:0$0:0", ...
  RET debug-location !25; t.c:7:1

*** Bad machine code: Explicit definition marked as use ***
- function:    test
- basic block: %bb.0 entry (0x6210000d8a90)
- instruction: CORE_MEM killed %4:gpr, 411, %0:gpr, @"llvm.t:0:0$0:0", ...
- operand 0:   killed %4:gpr

This happens because `CORE_MEM` instruction is defined to have output
operands:

  def CORE_MEM : TYPE_LD_ST<BPF_MEM.Value, BPF_W.Value,
                            (outs GPR:$dst),
                            (ins u64imm:$opcode, GPR:$src, u64imm:$offset),
                            "$dst = core_mem($opcode, $src, $offset)",
                            []>;

As documented in [1]:

> By convention, the LLVM code generator orders instruction operands
> so that all register definitions come before the register uses, even
> on architectures that are normally printed in other orders.

In other words, the first argument for `CORE_MEM` is considered to be
a "def", while in reality it is "use":

  %1:gpr = LD_imm64 @"llvm.t:0:0$0:0"
  %3:gpr = ADD_rr %0:gpr(tied-def 0), killed %1:gpr
  %4:gpr = MOV_ri 42
   '---------------.
                   v
  CORE_MEM killed %4:gpr, 411, %0:gpr, @"llvm.t:0:0$0:0", ...

Here is how `CORE_MEM` is constructed in
`BPFMISimplifyPatchable::checkADDrr()`:

    BuildMI(*DefInst->getParent(), *DefInst, DefInst->getDebugLoc(), TII->get(COREOp))
        .add(DefInst->getOperand(0)).addImm(Opcode).add(*BaseOp)
        .addGlobalAddress(GVal);

Note that first operand is constructed as `.add(DefInst->getOperand(0))`.

For `LD{D,W,H,B}` instructions the `DefInst->getOperand(0)` is a
destination register of a load, so instruction is constructed in
accordance with `outs` declaration.

For `ST{D,W,H,B}` instructions the `DefInst->getOperand(0)` is a
source register of a store (value to be stored), so instruction
violates the `outs` declaration.

This commit fixes the issue by splitting `CORE_MEM` in three
instructions: `CORE_ST`, `CORE_LD64`, `CORE_LD32` with correct `outs`
specifications.

[1] https://llvm.org/docs/CodeGenerator.html#the-machineinstr-class

Differential Revision: https://reviews.llvm.org/D157806

Enable vpu feature by default in the backend.

Reviewed By: efocht

Differential Revision: https://reviews.llvm.org/D157887

When LLVM is build with `LLVM_ENABLE_EXPENSIVE_CHECKS=ON` option
the following C code snippet:

    struct t {
      unsigned long a;
    } __attribute__((preserve_access_index));

    void foo(volatile struct t *t, volatile unsigned long *p) {
      *p = t->a;
      *p = t->a;
    }

Causes an assertion:

    $ clang -g -O2 -c --target=bpf -mcpu=v2 t2.c -o /dev/null

    # After BPF PreEmit SimplifyPatchable
    # Machine code for function foo: IsSSA, TracksLiveness
    Function Live Ins: $r1 in %0, $r2 in %1

    bb.0.entry:
      liveins: $r1, $r2
      DBG_VALUE $r1, $noreg, !"t", !DIExpression()
      DBG_VALUE $r2, $noreg, !"p", !DIExpression()
      %1:gpr = COPY $r2
      DBG_VALUE %1:gpr, $noreg, !"p", !DIExpression()
      %0:gpr = COPY $r1
      DBG_VALUE %0:gpr, $noreg, !"t", !DIExpression()
      %2:gpr = LD_imm64 @"llvm.t:0:0$0:0"
      %4:gpr = ADD_rr %0:gpr(tied-def 0), killed %2:gpr
      %5:gpr = CORE_LD 344, %0:gpr, @"llvm.t:0:0$0:0"
      STD killed %5:gpr, %1:gpr, 0
      %7:gpr = ADD_rr %0:gpr(tied-def 0), killed %2:gpr
      %8:gpr = CORE_LD 344, %0:gpr, @"llvm.t:0:0$0:0"
      STD killed %8:gpr, %1:gpr, 0
      RET

    # End machine code for function foo.

    *** Bad machine code: Using a killed virtual register ***
    - function:    foo
    - basic block: %bb.0 entry (0x6210000e6690)
    - instruction: %7:gpr = ADD_rr %0:gpr(tied-def 0), killed %2:gpr
    - operand 2:   killed %2:gpr

This happens because of the way
BPFMISimplifyPatchable::processDstReg() updates second operand of the
`ADD_rr` instruction. Code before `BPFMISimplifyPatchable`:

    .-> %2:gpr = LD_imm64 @"llvm.t:0:0$0:0"
    |
    |`----------------.
    |   %3:gpr = LDD %2:gpr, 0
    |   %4:gpr = ADD_rr %0:gpr(tied-def 0), killed %3:gpr <--- (1)
    |   %5:gpr = LDD killed %4:gpr, 0       ^^^^^^^^^^^^^
    |   STD killed %5:gpr, %1:gpr, 0        this is updated
     `----------------.
        %6:gpr = LDD %2:gpr, 0
        %7:gpr = ADD_rr %0:gpr(tied-def 0), killed %6:gpr <--- (2)
        %8:gpr = LDD killed %7:gpr, 0       ^^^^^^^^^^^^^
        STD killed %8:gpr, %1:gpr, 0        this is updated

Instructions (1) and (2) would be updated to:

    ADD_rr %0:gpr(tied-def 0), killed %2:gpr

The `killed` mark is inherited from machine operands `killed %3:gpr`
and `killed %6:gpr` which are updated inplace by `processDstReg()`.

This commit updates `processDstReg()` reset kill marks for updated
machine operands to keep liveness information conservatively correct.

Differential Revision: https://reviews.llvm.org/D157805

... that have the outliner enabled.

Differential revision: https://reviews.llvm.org/D157925

This reverts commit 2107d87c.

Change is causing test failures on many bots:
- https://lab.llvm.org/buildbot/#/builders/164/builds/42180
- https://lab.llvm.org/buildbot/#/builders/188/builds/33880
- https://lab.llvm.org/buildbot/#/builders/245/builds/12558
- https://lab.llvm.org/buildbot/#/builders/258/builds/5404
- https://lab.llvm.org/buildbot/#/builders/249/builds/8049
- https://lab.llvm.org/buildbot/#/builders/230/builds/17170
- https://lab.llvm.org/buildbot/#/builders/91/builds/17108
- https://lab.llvm.org/buildbot/#/builders/109/builds/71264
- https://lab.llvm.org/buildbot/#/builders/139/builds/47461
- https://lab.llvm.org/buildbot/#/builders/216/builds/25598
- https://lab.llvm.org/buildbot/#/builders/36/builds/36549
- https://lab.llvm.org/buildbot/#/builders/247/builds/7747

This reverts commit 2105c32b.

Reverting this follow-up commit as a prior commit (2107d87c) is causing build bot test failures.

Check iterator validity before use; fixes a crash seen in the RISC-V
Zcmp Push/Pop optimization pass when compiling an internal benchmark.

Reviewed By: asb, wangpc

Differential Revision: https://reviews.llvm.org/D157674

My primary motivation here is actually to change something in
UnixSignals, but this change is a necesary precondition.

I've also updated the documentation and rewritten the log statements to
use `formatv` instead of `printf` (printf-style formatting and
llvm::StringRef don't mix well).

Differential Revision: https://reviews.llvm.org/D157662

This reverts commit 7d06f59b.

This patch broke libc compilation with gcc as it doesn't seem to have
__is_trivially_destructible(T).

I somehow missed this path when adding the new expansions. Saves a lot
of instructions for afn + IEEE.

https://reviews.llvm.org/D157867

We should probably just swap this out for the fdiv, but that's what
the implementation is anyway.

These just get replaced with an intrinsic now. This was also
introducing host dependence on the result since it relied on the
compiler choice to contract or not.

This patch moves the storage from inside the libc's optional class to
its own set of class, so we can support non-trivially destructible
objects.

These new classes check if the class is or isn't non trivially
destructible and instantiate the correct base class, i.e., we explicitly
call the destructor if an object is not trivially destructible.

The motivation is to support cpp::optional<UInt<128>> (used by
UInt<T>::div), which is used when a platform does not support native
int128_t types (e.g., riscv32).

The code here is a trimmed-down version of llvm::optional.

Reviewed By: michaelrj

Differential Revision: https://reviews.llvm.org/D150211

Preserves flags and metadata like the other cases.

Turned out we were making overly simple assumptions about which sections (& section flags) would be used when emitting a global into a custom section. This lead to sections with read-only flags being used for globals of struct types with mutable members.

Fixed by porting the codegen function with the more nuanced handling/checking for mutable members out of codegen for use in the sema code that does this initial checking/mapping to section flags.

Differential Revision: https://reviews.llvm.org/D156726

OpenCL loses fast math information by going through libcall wrappers
around intrinsics.

Do this to preserve call site flags which are lost when inlining. It's
not safe in general to propagate flags during inline, so avoid dealing
with this by just special casing some of the useful calls.

It's failing on the Darwin CI: https://green.lab.llvm.org/green/ since it was introduced by https://reviews.llvm.org/D157552 and the failure hasn't been resolved yet

Previous fix attempt (beae3152) did not resolve the failure, so marking it as unsupported again.

rdar://113765281

Consistent order of ops and related methods.
Also, renamed SpGEMMGetSizeOp to SpMatGetSizeOp
since this is a general utility for sparse matrices,
not specific to GEMM ops only.

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D157922

Reviewed By: jhenderson, MaskRay
Differential Revision: https://reviews.llvm.org/D149095

I think after making G_SEXT_INREG legal this isn't needed. At the
very least its not tested anymore.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D157678

If we lower, we need to legalize the wide shifts which
is costly.

This will improve the tests from https://reviews.llvm.org/D157415 too

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D157677

As stated on Discourse*, these methods have been deprecated. I am
removing their implementation. They will now do nothing and return a
value indicating failure (where appropriate).
Due to the LLDB project's commitment to ABI stability at the SB API
layer, we cannot remove these symbols completely.

Discourse link: https://discourse.llvm.org/t/do-you-use-the-threading-functionality-in-sbhostos/71973

This is not an FP32 operation.

Differential Revision: https://reviews.llvm.org/D157909

These were useful primarily for the Python 2 to 3 transition. Python 2
is no longer supported so these are no longer necessary.

Differential Revision: https://reviews.llvm.org/D157759

Fixed comparator for PHI nodes sorting to meet the criteria for strict
weak ordering.

Try to avoid some unprofitable predication on PPC. Recognize in the cost model that computing on i1 values will require extra mask or compare operation.

Differential Revision: https://reviews.llvm.org/D155876

Differential Revision: https://reviews.llvm.org/D157814

Differential Revision: https://reviews.llvm.org/D157808