aboutsummaryrefslogtreecommitdiff
path: root/llvm/lib/CodeGen/MachineBasicBlock.cpp
AgeCommit message (Collapse)AuthorFilesLines
2024-04-17[LiveIns] Improve recomputeLiveIns() (#88951)Kai Nacke1-0/+6
Some small changes to recomputeLiveIns() to improve performance: - Instead of copying the list of old live-ins, and then clearing them, a new method swaps the list for an empty one. - getLiveIns() now returns a constant reference to the list As result, the list-data is never copied. Depending on the implementation details of the vector container, it can also save calls to allocate and deallocate memory. I see a small improvement on CTMark with these changes. --------- Co-authored-by: Nikita Popov <github@npopov.com>
2024-04-08[CodeGen][DebugInfo] Add missing DebugLoc for SplitCriticalEdge (#72192)Haohai Wen1-1/+9
In SplitCriticalEdge, DebugLoc of the branch instruction in new created MBB was set to empty. It should be set and we can find proper DebugLoc for it in most cases. This patch set it to non empty merged DebugLoc of current MBB branches.
2023-11-16[AMDGPU] RA inserted scalar instructions can be at the BB top (#72140)Christudasan Devadasan1-2/+2
We adjust the insertion point at the BB top for spills/copies during RA to ensure they are placed after the exec restore instructions required for the divergent control flow execution. This is, however, required only for the vector operations. The insertions for scalar registers can still go to the BB top.
2023-11-13[PHIElimination] Handle subranges in LiveInterval updates (#69429)Carl Ritson1-0/+10
Add subrange tracking and handling for LiveIntervals during PHI elimination. This requires extending MachineBasicBlock::SplitCriticalEdge to also update subrange intervals.
2023-10-27[BasicBlockSections] Apply path cloning with -basic-block-sections. (#68860)Rahman Lavaee1-2/+4
https://github.com/llvm/llvm-project/commit/28b912687900bc0a67cd61c374fce296b09963c4 introduced the path cloning format in the basic-block-sections profile. This PR validates and applies path clonings. A path cloning is valid if all of these conditions hold: 1. All bb ids in the path are mapped to existing blocks. 2. Each two consecutive bb ids in the path have a successor relationship in the CFG. 3. The path does not include a block with indirect branches, except possibly as the last block. Applying a path cloning involves cloning all blocks in the path (except the first one) and setting up their branches. Once all clonings are applied, the cluster information is used to guide block layout in the modified function.
2023-10-21[MachineBasicBlock] Fix SlotIndexUpdater for insertion order (#69424)Carl Ritson1-3/+9
Follow up fix for #68786 to address that MachineFunction handleInsertion is actually called before a new instruction has been inserted into the block. Hence new instructions must be recorded and SlotIndex updates performed after the delegate call.
2023-10-15[MachineBasicBlock] Fix use after free in SplitCriticalEdge (#68786)Carl Ritson1-31/+27
Remove use after free when attempting to update SlotIndexes in MachineBasicBlock::SplitCriticalEdge. Use MachineFunction delegate mechanism to capture target specific manipulations of branch instructions and update SlotIndexes.
2023-10-06AMDGPU: Fix temporal divergence introduced by machine-sink (#67456)Petar Avramovic1-0/+4
Temporal divergence that was present in input or introduced in IR transforms, like code-sinking or LICM, is handled in SIFixSGPRCopies by changing sgpr source instr to vgpr instr. After 5b657f5, that moved LICM after AMDGPUCodeGenPrepare, machine-sinking can introduce temporal divergence by sinking instructions outside of the cycle. Add isSafeToSink callback in TargetInstrInfo.
2023-09-11Revert "[PHIElimination] Handle subranges in LiveInterval updates"Vitaly Buka1-12/+0
Leaks memory. This reverts commit 3bff611068ae70e3273a46bbc72bc66b66f98c1c.
2023-09-11[PHIElimination] Handle subranges in LiveInterval updatesCarl Ritson1-0/+12
Add handling for subrange updates in LiveInterval preservation. This requires extending MachineBasicBlock::SplitCriticalEdge to also update subrange intervals. Reviewed By: arsenm Differential Revision: https://reviews.llvm.org/D158144
2023-08-20[Propeller] Deprecate Codegen paths for SHT_LLVM_BB_ADDR_MAP version 1.Rahman Lavaee1-5/+0
This patch removes the `getBBIDOrNumber` which was introduced to allow emitting version 1. Reviewed By: shenhan Differential Revision: https://reviews.llvm.org/D158299
2023-07-27[CodeGen] Store call frame size in MachineBasicBlockJay Foad1-0/+6
Record the call frame size on entry to each basic block. This is usually zero except when a basic block has been split in the middle of a call sequence. This simplifies PEI::replaceFrameIndices which previously had to visit basic blocks in a specific order and had special handling for unreachable blocks. More importantly it paves the way for an equally simple implementation of a backwards version of replaceFrameIndices, which is required to fully convert PrologEpilogInserter to backwards register scavenging, which is preferred because it does not rely on accurate kill flags. Differential Revision: https://reviews.llvm.org/D156113
2023-07-13Revert "[CodeGen] Store SP adjustment in MachineBasicBlock. NFCI."Oliver Stannard1-5/+0
This reverts commit 58d1eaa3b6ce4f7285c51f83faff7a3ac374c746.
2023-07-12[CodeGen] Store SP adjustment in MachineBasicBlock. NFCI.Jay Foad1-0/+5
Record the SP adjustment on entry to each basic block. This is almost always zero except on targets like ARM which can split a basic block in the middle of a call sequence. This simplifies PEI::replaceFrameIndices which previously had to visit basic blocks in a specific order and had special handling for unreachable blocks. More importantly it paves the way for an equally simple implementation of a backwards version of replaceFrameIndices, which is required to fully convert PrologEpilogInserter to backwards register scavenging, which is preferred because it does not rely on accurate kill flags. Differential Revision: https://reviews.llvm.org/D154281
2023-06-25[llvm] Add missing StringExtras.h includesElliot Goodrich1-0/+1
In preparation for removing the `#include "llvm/ADT/StringExtras.h"` from the header to source file of `llvm/Support/Error.h`, first add in all the missing includes that were previously included transitively through this header.
2023-06-01[CodeGen] Make use of MachineInstr::all_defs and all_uses. NFCI.Jay Foad1-3/+2
Differential Revision: https://reviews.llvm.org/D151424
2023-05-25[CodeGen] Fix for MachineBasicBlock::rfindDebugLoc(instr_rend())Bjorn Pettersson1-0/+2
Make sure we do not crash in rfindDebugLoc when starting at instr_rend(). Solution is to see it as we start one MI before the first MI, so we can start searching forward at instr_begin() instead. This behavior is similar to how findPrevDebugLoc(instr_end()) works. Differential Revision: https://reviews.llvm.org/D150577
2023-05-25[CodeGen] Add unittest for findDebugLoc, rfindDebugLoc, findPrevDebugLoc and ↵Bjorn Pettersson1-4/+6
rfindPrevDebugLoc. NFC - Add some unittests for the findDebugLoc, rfindDebugLoc, findPrevDebugLoc and rfindPrevDebugLoc helpers in MachineBasicBlock. - Clean up code comments and code formatting related to the functions mentioned above. This was extracted as a pre-commit to D150577, adn some of the tests are commented out since they would crash/assert in a rather uncontrolled way.
2023-05-10Support critical edge splitting for jump tablesMatthias Braun1-2/+73
Add support for splitting critical edges coming from an indirect jump using a jump table ("switch jumps"). This introduces the `TargetInstrInfo::getJumpTableIndex` callback to allows targets to return an index into `MachineJumpTableInfo` for a given indirect jump. It also updates to `MachineBasicBlock::SplitCriticalEdge` to allow splitting of critical edges by rewriting jump table entries. This is largely based on work done by Zhixuan Huan in D132202. Differential Revision: https://reviews.llvm.org/D140975
2023-04-12[BranchRelaxation] Correct JumpToFT valueAnshil Gandhi1-2/+2
Toggle true/false values of the JumpToFallThrough parameter to simplify code and make it consistent with the documentation for the `getFallThrough(..)` method. Reviewed By: bcahoon Differential Revision: https://reviews.llvm.org/D148139
2023-01-17[MachineBasicBlock] Explicit FT branching paramAnshil Gandhi1-3/+3
Introduce a parameter in getFallThrough() to optionally allow returning the fall through basic block in spite of an explicit branch instruction to it. This parameter is set to false by default. Introduce getLogicalFallThrough() which calls getFallThrough(false) to obtain the block while avoiding insertion of a jump instruction to its immediate successor. This patch also reverts the changes made by D134557 and solves the case where a jump is inserted after another jump (branch-relax-no-terminators.mir). Reviewed By: arsenm Differential Revision: https://reviews.llvm.org/D140790
2023-01-17[Propeller] Use Fixed MBB ID instead of volatile MachineBasicBlock::Number.Rahman Lavaee1-0/+10
Let Propeller use specialized IDs for basic blocks, instead of MBB number. This allows optimizations not just prior to asm-printer, but throughout the entire codegen. This patch only implements the functionality under the new `LLVM_BB_ADDR_MAP` version, but the old version is still being used. A later patch will change the used version. ####Background Today Propeller uses machine basic block (MBB) numbers, which already exist, to map native assembly to machine IR. This is done as follows. - Basic block addresses are captured and dumped into the `LLVM_BB_ADDR_MAP` section just before the AsmPrinter pass which writes out object files. This ensures that we have a mapping that is close to assembly. - Profiling mapping works by taking a virtual address of an instruction and looking up the `LLVM_BB_ADDR_MAP` section to find the MBB number it corresponds to. - While this works well today, we need to do better when we scale Propeller to target other Machine IR optimizations like spill code optimization. Register allocation happens earlier in the Machine IR pipeline and we need an annotation mechanism that is valid at that point. - The current scheme will not work in this scenario because the MBB number of a particular basic block is not fixed and changes over the course of codegen (via renumbering, adding, and removing the basic blocks). - In other words, the volatile MBB numbers do not provide a one-to-one correspondence throughout the lifetime of Machine IR. Profile annotation using MBB numbers is restricted to a fixed point; only valid at the exact point where it was dumped. - Further, the object file can only be dumped before AsmPrinter and cannot be dumped at an arbitrary point in the Machine IR pass pipeline. Hence, MBB numbers are not suitable and we need something else. ####Solution We propose using fixed unique incremental MBB IDs for basic blocks instead of volatile MBB numbers. These IDs are assigned upon the creation of machine basic blocks. We modify `MachineFunction::CreateMachineBasicBlock` to assign the fixed ID to every newly created basic block. It assigns `MachineFunction::NextMBBID` to the MBB ID and then increments it, which ensures having unique IDs. To ensure correct profile attribution, multiple equivalent compilations must generate the same Propeller IDs. This is guaranteed as long as the MachineFunction passes run in the same order. Since the `NextBBID` variable is scoped to `MachineFunction`, interleaving of codegen for different functions won't cause any inconsistencies. The new encoding is generated under the new version number 2 and we keep backward-compatibility with older versions. ####Impact on Size of the `LLVM_BB_ADDR_MAP` Section Emitting the Propeller ID results in a 23% increase in the size of the `LLVM_BB_ADDR_MAP` section for the clang binary. Reviewed By: tmsriram Differential Revision: https://reviews.llvm.org/D100808
2023-01-13[CodeGen] Remove uses of Register::isPhysicalRegister/isVirtualRegister. NFCCraig Topper1-3/+2
Use isPhysical/isVirtual methods. Reviewed By: foad Differential Revision: https://reviews.llvm.org/D141715
2022-12-16[Transforms,CodeGen] std::optional::value => operator*/operator->Fangrui Song1-2/+2
value() has undesired exception checking semantics and calls __throw_bad_optional_access in libc++. Moreover, the API is unavailable without _LIBCPP_NO_EXCEPTIONS on older Mach-O platforms (see _LIBCPP_AVAILABILITY_BAD_OPTIONAL_ACCESS).
2022-12-13Revert "[Propeller] Use Fixed MBB ID instead of volatile ↵Rahman Lavaee1-10/+0
MachineBasicBlock::Number." This reverts commit 6015a045d768feab3bae9ad9c0c81e118df8b04a. Differential Revision: https://reviews.llvm.org/D139952
2022-12-07[GlobalISel] Add a new G_INVOKE_REGION_START instruction to fix an EH bug.Amara Emerson1-0/+5
We currently have a bug where the legalizer, when dealing with phi operands, may create instructions in the phi's incoming blocks at points which are effectively dead due to a possible exception throw. Say we have: throwbb: EH_LABEL x0 = %callarg1 BL @may_throw_call EH_LABEL B returnbb bb: %v = phi i1 %true, throwbb, %false.... When legalizing we may need to widen the i1 %true value, and to do that we need to create new extension instructions in the incoming block. Our insertion point currently is the MBB::getFirstTerminator() which puts the IP before the unconditional branch terminator in throwbb. These extensions may never be executed if the call throws, and therefore we need to emit them before the call (but not too early, since our new instruction may need values defined within throwbb as well). throwbb: EH_LABEL x0 = %callarg1 BL @may_throw_call EH_LABEL %true = G_CONSTANT i32 1 ; <<<-- ruh'roh, this never executes if may_throw_call() throws! B returnbb bb: %v = phi i32 %true, throwbb, %false.... To fix this, I've added two new instructions. The main idea is that G_INVOKE_REGION_START is a terminator, which tries to model the fact that in the IR, the original invoke inst is actually a terminator as well. By using that as the new insertion point, we make sure to place new instructions on always executing paths. Unfortunately we still need to make the legalizer use a new insertion point API that I've added, since the existing `getFirstTerminator()` method does a reverse walk up the block, and any non-terminator instructions cause it to bail out. To avoid impacting compile time for all `getFirstTerminator()` uses, I've added a new method that does a forward walk instead. Differential Revision: https://reviews.llvm.org/D137905
2022-12-06[Propeller] Use Fixed MBB ID instead of volatile MachineBasicBlock::Number.Rahman Lavaee1-0/+10
Let Propeller use specialized IDs for basic blocks, instead of MBB number. This allows optimizations not just prior to asm-printer, but throughout the entire codegen. This patch only implements the functionality under the new `LLVM_BB_ADDR_MAP` version, but the old version is still being used. A later patch will change the used version. ####Background Today Propeller uses machine basic block (MBB) numbers, which already exist, to map native assembly to machine IR. This is done as follows. - Basic block addresses are captured and dumped into the `LLVM_BB_ADDR_MAP` section just before the AsmPrinter pass which writes out object files. This ensures that we have a mapping that is close to assembly. - Profiling mapping works by taking a virtual address of an instruction and looking up the `LLVM_BB_ADDR_MAP` section to find the MBB number it corresponds to. - While this works well today, we need to do better when we scale Propeller to target other Machine IR optimizations like spill code optimization. Register allocation happens earlier in the Machine IR pipeline and we need an annotation mechanism that is valid at that point. - The current scheme will not work in this scenario because the MBB number of a particular basic block is not fixed and changes over the course of codegen (via renumbering, adding, and removing the basic blocks). - In other words, the volatile MBB numbers do not provide a one-to-one correspondence throughout the lifetime of Machine IR. Profile annotation using MBB numbers is restricted to a fixed point; only valid at the exact point where it was dumped. - Further, the object file can only be dumped before AsmPrinter and cannot be dumped at an arbitrary point in the Machine IR pass pipeline. Hence, MBB numbers are not suitable and we need something else. ####Solution We propose using fixed unique incremental MBB IDs for basic blocks instead of volatile MBB numbers. These IDs are assigned upon the creation of machine basic blocks. We modify `MachineFunction::CreateMachineBasicBlock` to assign the fixed ID to every newly created basic block. It assigns `MachineFunction::NextMBBID` to the MBB ID and then increments it, which ensures having unique IDs. To ensure correct profile attribution, multiple equivalent compilations must generate the same Propeller IDs. This is guaranteed as long as the MachineFunction passes run in the same order. Since the `NextBBID` variable is scoped to `MachineFunction`, interleaving of codegen for different functions won't cause any inconsistencies. The new encoding is generated under the new version number 2 and we keep backward-compatibility with older versions. ####Impact on Size of the `LLVM_BB_ADDR_MAP` Section Emitting the Propeller ID results in a 23% increase in the size of the `LLVM_BB_ADDR_MAP` section for the clang binary. Reviewed By: tmsriram Differential Revision: https://reviews.llvm.org/D100808
2022-09-28[iwyu] Move <cmath> out of llvm/Support/MathExtras.hserge-sans-paille1-0/+1
Interestingly, MathExtras.h doesn't use <cmath> declaration, so move it out of that header and include it when needed. No functional change intended, but there's no longer a transitive include fromMathExtras.h to cmath.
2022-08-16Untangle the mess which is MachineBasicBlock::hasAddressTaken().Eli Friedman1-17/+31
There are two different senses in which a block can be "address-taken". There can be a BlockAddress involved, which means we need to map the IR-level value to some specific block of machine code. Or there can be constructs inside a function which involve using the address of a basic block to implement certain kinds of control flow. Mixing these together causes a problem: if target-specific passes are marking random blocks "address-taken", if we have a BlockAddress, we can't actually tell which MachineBasicBlock corresponds to the BlockAddress. So split this into two separate bits: one for BlockAddress, and one for the machine-specific bits. Discovered while trying to sort out related stuff on D102817. Differential Revision: https://reviews.llvm.org/D124697
2022-07-17[CodeGen] Qualify auto variables in for loops (NFC)Kazu Hirata1-1/+1
2022-07-13[llvm] Use value instead of getValue (NFC)Kazu Hirata1-1/+1
2022-06-25Revert "Don't use Optional::hasValue (NFC)"Kazu Hirata1-2/+2
This reverts commit aa8feeefd3ac6c78ee8f67bf033976fc7d68bc6d.
2022-06-25Don't use Optional::hasValue (NFC)Kazu Hirata1-2/+2
2022-06-23[AMDGPU] Flush the vmcnt counter in loop preheaders when necessaryBaptiste Saleil1-0/+4
waitcnt vmcnt instructions are currently generated in loop bodies before using values loaded outside of the loop. In some cases, it is better to flush the vmcnt counter in a loop preheader before entering the loop body. This patch detects these cases and generates waitcnt instructions to flush the counter. Reviewed By: foad Differential Revision: https://reviews.llvm.org/D115747
2022-06-21[machinesink] fix debug invariance issueMarkus Lavin1-0/+10
Do not include debug instructions when comparing block sizes with thresholds. Differential Revision: https://reviews.llvm.org/D127208
2022-03-16[NFC][CodeGen] Rename some functions in MachineInstr.h and remove duplicated ↵Shengchen Kan1-3/+3
comments
2022-03-16Cleanup codegen includesserge-sans-paille1-3/+1
This is a (fixed) recommit of https://reviews.llvm.org/D121169 after: 1061034926 before: 1063332844 Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup Differential Revision: https://reviews.llvm.org/D121681
2022-03-10Revert "Cleanup codegen includes"Nico Weber1-1/+3
This reverts commit 7f230feeeac8a67b335f52bd2e900a05c6098f20. Breaks CodeGenCUDA/link-device-bitcode.cu in check-clang, and many LLVM tests, see comments on https://reviews.llvm.org/D121169
2022-03-10Cleanup codegen includesserge-sans-paille1-3/+1
after: 1061034926 before: 1063332844 Differential Revision: https://reviews.llvm.org/D121169
2022-02-06[CodeGen] Use = default (NFC)Kazu Hirata1-2/+1
Identified with modernize-use-equals-default
2021-12-08[NFC] Rename MachineFunction::deleteMachineInstr (coding style)Mircea Trofin1-1/+1
2021-12-06[CodeGen] Use range-based for loops (NFC)Kazu Hirata1-16/+13
2021-11-16[llvm] Use range-for loops (NFC)Kazu Hirata1-3/+2
2021-11-11[CodeGen] Use MachineInstr::operands (NFC)Kazu Hirata1-11/+9
2021-11-09[llvm] Use MachineBasicBlock::{successors,predecessors} (NFC)Kazu Hirata1-2/+2
2021-10-07[MIRParser] Add support for IsInlineAsmBrIndirectTargetJay Foad1-0/+5
Print this basic block flag as inlineasm-br-indirect-target and parse it. This allows you to write MIR test cases for INLINEASM_BR. The test case I added is one that I wanted to precommit anyway for D110834. Differential Revision: https://reviews.llvm.org/D111291
2021-06-18[CSSPGO] Undoing the concept of dangling pseudo probeHongtao Yu1-26/+0
As a follow-up to https://reviews.llvm.org/D104129, I'm cleaning up the danling probe related code in both the compiler and llvm-profgen. I'm seeing a 5% size win for the pseudo_probe section for SPEC2017 and 10% for Ciner. Certain benchmark such as 602.gcc has a 20% size win. No obvious difference seen on build time for SPEC2017 and Cinder. Reviewed By: wenlei Differential Revision: https://reviews.llvm.org/D104477
2021-05-19MachineBasicBlock: add liveout iterator aware of which liveins are defined ↵Tim Northover1-0/+18
by the runtime. Using this in RegAlloc fast reduces register pressure, and in some cases allows x86 code to compile that wouldn't before.
2021-04-19[CSSPGO] Exclude pseudo probes from slot indexHongtao Yu1-3/+3
Pseudo probe are currently given a slot index like other regular instructions. This affects register pressure and lifetime weight computation because of enlarged lifetime length with pseudo probe instructions. As a consequence, program could get different code generated w/ and w/o pseudo probes. I'm closing the gap by excluding pseudo probes from stack index and downstream register allocation related passes. Reviewed By: wmi Differential Revision: https://reviews.llvm.org/D100334
2021-04-19[CSSPGO] Flip SkipPseudoOp to true for MIR APIs.Hongtao Yu1-1/+3
Flipping the default value of SkipPseudoOp to true for those MIR APIs to favor maximum performance. Note that certain spots like branch folding and MIR if-conversion is are disabled for better counts quality. For these two optimizations, this is a no-diff change. The counts quality with SPEC2017 before/after this change is unchanged. Reviewed By: wmi Differential Revision: https://reviews.llvm.org/D100332
generated by cgit v1.1 at 2025-10-03 23:14:12 +0000