aboutsummaryrefslogtreecommitdiff
path: root/llvm/lib/CodeGen/MachineSink.cpp
AgeCommit message (Collapse)AuthorFilesLines
2023-09-29[AArch64] Fix a compiler crash in MachineSink (#67705)Momchil Velikov1-17/+19
There were a couple of issues with maintaining register def/uses held in `MachineRegisterInfo`: * when an operand is changed from one register to another, the corresponding instruction must already be inserted into the function, or MRI won't be updated * when traversing the set of all uses of a register, that set must not change
2023-09-25[MachineSink][AArch64] Sink instruction copies when they can replace copy ↵Momchil Velikov1-29/+268
into hard register or folded into addressing mode This patch adds a new code transformation to the `MachineSink` pass, that tries to sink copies of an instruction, when the copies can be folded into the addressing modes of load/store instructions, or replace another instruction (currently, copies into a hard register). The criteria for performing the transformation is that: * the register pressure at the sink destination block must not exceed the register pressure limits * the latency and throughput of the load/store or the copy must not deteriorate * the original instruction must be deleted Reviewed By: dmgreen Differential Revision: https://reviews.llvm.org/D152828
2023-08-14[CodeGen] Set regunitmasks for leaf regs to all instead of noneJay Foad1-3/+2
This simplifies every use of MCRegUnitMaskIterator. Differential Revision: https://reviews.llvm.org/D157864
2023-08-09Remove a reference to rdar://problem/8030636Jon Roelofs1-1/+1
The surrounding comment has more than enough context to describe the problem.
2023-08-02MachineSink: Fix strict weak ordering in GetAllSortedSuccessorsDanila Kutenin1-1/+1
CodeGen/X86/pseudo_cmov_lower2.ll fails using libc++ debug mode (D150264) without this change. Reviewed By: MaskRay, aeubanks Differential Revision: https://reviews.llvm.org/D155811
2023-07-18MachineSink: Fix sinking VGPR def out of a divergent loopMatt Arsenault1-4/+11
This fixes sinking a VGPR def out of a loop past the reconvergence point at the SI_END_CF. There was a prior fix which introduced blockPrologueInterferes (D121277) to fix the same basic problem for the post RA sink. This also had the special case isIgnorableUse case which was incorrect, because in some contexts the exec use is not ignorable. I'm thinking about a new way to represent this which will avoid needing hasIgnorableUse and isBasicBlockPrologue, which would function more like the exception handling. Fixes: SWDEV-407790 https://reviews.llvm.org/D155343
2023-07-14MachineSink: Remove unnecessary empty block checkMatt Arsenault1-2/+0
2023-07-14MachineSink: Move helper function and use more constMatt Arsenault1-39/+41
2023-06-16[MC] Add MCRegisterInfo::regunits for iteration over register unitsSergei Barannikov1-5/+4
Reviewed By: foad Differential Revision: https://reviews.llvm.org/D152098
2023-06-01[CodeGen] Make use of MachineInstr::all_defs and all_uses. NFCI.Jay Foad1-18/+9
Differential Revision: https://reviews.llvm.org/D151424
2023-05-16[MachineSink] Don't reject sinking because of dead def in ↵Jonas Paulsson1-6/+4
isProfitableToSinkTo(). An instruction should be sunk (if otherwise legal and profitable) regardless of if it has a dead def of a physreg or not. Physreg defs are checked in other places and sinking is only done with dead defs of regs that are not live into the target MBB. Differential Revision: https://reviews.llvm.org/D150447 Reviewed By: sebastian-ne, arsenm
2023-04-18[MC] Use subregs/superregs instead of MCSubRegIterator/MCSuperRegIterator. NFC.Jay Foad1-2/+2
Differential Revision: https://reviews.llvm.org/D148613
2023-04-17Fix uninitialized pointer members in CodeGenAkshay Khadse1-9/+9
This change initializes the members TSI, LI, DT, PSI, and ORE pointer feilds of the SelectOptimize class to nullptr. Reviewed By: LuoYuanke Differential Revision: https://reviews.llvm.org/D148303
2023-02-07[CodeGen] Define and use MachineOperand::getOperandNoJay Foad1-1/+1
This is a helper function to very slightly simplify many calls to MachineInstruction::getOperandNo. Differential Revision: https://reviews.llvm.org/D143250
2023-01-13[CodeGen] Remove uses of Register::isPhysicalRegister/isVirtualRegister. NFCCraig Topper1-9/+9
Use isPhysical/isVirtual methods. Reviewed By: foad Differential Revision: https://reviews.llvm.org/D141715
2023-01-06[DebugInfo][NFC] Add new MachineOperand type and change DBG_INSTR_REF syntaxStephen Tozer1-1/+1
This patch makes two notable changes to the MIR debug info representation, which result in different MIR output but identical final DWARF output (NFC w.r.t. the full compilation). The two changes are: * The introduction of a new MachineOperand type, MO_DbgInstrRef, which consists of two unsigned numbers that are used to index an instruction and an output operand within that instruction, having a meaning identical to first two operands of the current DBG_INSTR_REF instruction. This operand is only used in DBG_INSTR_REF (see below). * A change in syntax for the DBG_INSTR_REF instruction, shuffling the operands to make it resemble DBG_VALUE_LIST instead of DBG_VALUE, and replacing the first two operands with a single MO_DbgInstrRef-type operand. This patch is the first of a set that will allow DBG_INSTR_REF instructions to refer to multiple machine locations in the same manner as DBG_VALUE_LIST. Reviewed By: jmorse Differential Revision: https://reviews.llvm.org/D129372
2022-07-17[CodeGen] Qualify auto variables in for loops (NFC)Kazu Hirata1-1/+1
2022-06-24[MachineSink] Clear kill flags on operands outside loopCarl Ritson1-0/+6
If an instruction is sunk into a loop then any kill flags on operands declared outside the loop must be cleared as these will be live for all loop iterations. Fixes #46827 Reviewed By: MatzeB Differential Revision: https://reviews.llvm.org/D126754
2022-06-21[machinesink] fix debug invariance issueMarkus Lavin1-2/+2
Do not include debug instructions when comparing block sizes with thresholds. Differential Revision: https://reviews.llvm.org/D127208
2022-06-15[CodeGen] Fix the bug of machine sinkLuo, Yuanke1-0/+2
The use operand may be undefined. In that case we can just continue to check the next operand since it won't increase register pressure. Differential Revision: https://reviews.llvm.org/D127848
2022-05-26[MachineSink] replace MachineLoop with MachineCycleChen Zheng1-94/+96
reapply 62a9b36fcf728b104ea87e6eb84c0be69b779df7 and fix module build failue: 1: remove MachineCycleInfoWrapperPass in MachinePassRegistry.def MachineCycleInfoWrapperPass is a anylysis pass, should not be there. 2: move the definition for MachineCycleInfoPrinterPass to cpp file. Otherwise, there are module conflicit for MachineCycleInfoWrapperPass in MachinePassRegistry.def and MachineCycleAnalysis.h after 62a9b36fcf728b104ea87e6eb84c0be69b779df7. MachineCycle can handle irreducible loop. Natural loop analysis (MachineLoop) can not return correct loop depth if the loop is irreducible loop. And MachineSink is sensitive to the loop depth, see MachineSinking::isProfitableToSinkTo(). This patch tries to use MachineCycle so that we can handle irreducible loop better. Reviewed By: sameerds, MatzeB Differential Revision: https://reviews.llvm.org/D123995
2022-05-24Revert "[MachineSink] replace MachineLoop with MachineCycle"Chen Zheng1-96/+94
This reverts commit 62a9b36fcf728b104ea87e6eb84c0be69b779df7. Cause build failure on lldb incremental buildbot: https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/43994/changes
2022-05-24[MachineSink] replace MachineLoop with MachineCycleChen Zheng1-94/+96
MachineCycle can handle irreducible loop. Natural loop analysis (MachineLoop) can not return correct loop depth if the loop is irreducible loop. And MachineSink is sensitive to the loop depth, see MachineSinking::isProfitableToSinkTo(). This patch tries to use MachineCycle so that we can handle irreducible loop better. Reviewed By: sameerds, MatzeB Differential Revision: https://reviews.llvm.org/D123995
2022-03-22[MachineSink] Check block prologue interferenceCarl Ritson1-2/+53
Sinking must check for interference between the block prologue and the instruction being sunk. Specifically check for clobbering of uses by the prologue, and overwrites to prologue defined registers by the sunk instruction. Reviewed By: rampitec, ruiling Differential Revision: https://reviews.llvm.org/D121277
2022-03-16Cleanup codegen includesserge-sans-paille1-1/+1
This is a (fixed) recommit of https://reviews.llvm.org/D121169 after: 1061034926 before: 1063332844 Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup Differential Revision: https://reviews.llvm.org/D121681
2022-03-10Revert "Cleanup codegen includes"Nico Weber1-1/+1
This reverts commit 7f230feeeac8a67b335f52bd2e900a05c6098f20. Breaks CodeGenCUDA/link-device-bitcode.cu in check-clang, and many LLVM tests, see comments on https://reviews.llvm.org/D121169
2022-03-10Cleanup codegen includesserge-sans-paille1-1/+1
after: 1061034926 before: 1063332844 Differential Revision: https://reviews.llvm.org/D121169
2022-03-02[MachineSink] Disable if there are any irreducible cyclesNikita Popov1-0/+12
This is an alternative to D120330, which disables MachineSink for functions with irreducible cycles entirely. This avoids both the correctness problem, and ensures we don't perform non-profitable sinks into cycles. At the same time, it may also disable profitable sinks in the same function. This can be made more precise by using MachineCycleInfo in the future. Fixes https://github.com/llvm/llvm-project/issues/53990. Differential Revision: https://reviews.llvm.org/D120800
2022-02-16[MachineSink] Use SkipPHIsAndLabels for sink insertion pointsCarl Ritson1-5/+6
For AMDGPU the insertion point for a block may not be the first non-PHI instruction. This happens when a block contains EXEC mask manipulation related to control flow (converging lanes). Use SkipPHIsAndLabels to determine the block insertion point so that the target can skip any block prologue instructions. Reviewed By: rampitec, ruiling Differential Revision: https://reviews.llvm.org/D119399
2022-02-12[MachineSink] Inline getRegUnitsBenjamin Kramer1-21/+11
Reg unit sets are uniqued, so no need to wrap it in a set.
2022-01-18[MachineSink] Allow sinking of constant or ignorable physreg usesVang Thao1-3/+8
For AMDGPU, any use of the physical register EXEC prevents sinking even if it is not a real physical register read. Add check to see if a physical register use can be ignored for sinking. Also perform same constant and ignorable physical register check when considering sinking in loops. https://reviews.llvm.org/D116053
2021-11-25[llvm] Use range-based for loops (NFC)Kazu Hirata1-8/+4
2021-11-12Fix minor deficiency in machine-sink.Markus Lavin1-1/+2
Register uses that are MRI->isConstantPhysReg() should not inhibit sinking transformation. Reviewed By: StephenTozer Differential Revision: https://reviews.llvm.org/D111531
2021-11-09[llvm] Use MachineBasicBlock::{successors,predecessors} (NFC)Kazu Hirata1-6/+2
2021-11-06[llvm] Use llvm::reverse (NFC)Kazu Hirata1-2/+1
2021-10-31[CodeGen] Use make_early_inc_range (NFC)Kazu Hirata1-19/+16
2021-10-18[MachineSink] Compile time improvement for large testcases which has many ↵Bing1 Yu1-2/+2
kill flags We did a experiment and observed dramatic decrease on compilation time which spent on clearing kill flags. Before: Number of BasicBlocks:33357 Number of Instructions:162067 Number of Cleared Kill Flags:32869 Time of handling kill flags(ms):1.607509e+05 After: Number of BasicBlocks:33357 Number of Instructions:162067 Number of Cleared Kill Flags:32869 Time of handling kill flags:3.987371e+03 Reviewed By: MatzeB Differential Revision: https://reviews.llvm.org/D111688
2021-07-08[DebugInfo][InstrRef][4/4] Support DBG_INSTR_REF through all backend passesJeremy Morse1-1/+1
This is a cleanup patch -- we're now able to support all flavours of variable location in instruction referencing mode. This patch updates various tests for debug instructions to be broader: numerous code paths try to ignore debug isntructions, and they now have to ignore the additional DBG_PHI and DBG_INSTR_REFs that we can generate. A small amount of rework happens for LiveDebugVariables: as we don't need to track live intervals through regalloc any more, we can get away with unlinking debug instructions before regalloc, then re-inserting them after. Note that this isn't (yet) true of DBG_VALUE_LISTs, they still have to go through live interval tracking. In SelectionDAG, add a helper lambda that emits half-formed DBG_INSTR_REFs for arguments in instr-ref mode, DBG_VALUE otherwise. This is one of the final locations where DBG_VALUEs are emitted for vreg arguments. X86InstrInfo now un-sets the debug instr number on SUB instructions that get mutated into CMP instructions. As the instruction no longer computes a subtraction, we can't use it for variable locations. Differential Revision: https://reviews.llvm.org/D88898
2021-04-19[CSSPGO] Exclude pseudo probes from slot indexHongtao Yu1-3/+3
Pseudo probe are currently given a slot index like other regular instructions. This affects register pressure and lifetime weight computation because of enlarged lifetime length with pseudo probe instructions. As a consequence, program could get different code generated w/ and w/o pseudo probes. I'm closing the gap by excluding pseudo probes from stack index and downstream register allocation related passes. Reviewed By: wmi Differential Revision: https://reviews.llvm.org/D100334
2021-03-10[DebugInfo] Handle DBG_VALUES with multiple variable location operands in MIRStephen Tozer1-43/+82
This patch adds handling for DBG_VALUE_LIST in the MIR-passes (after finalize-isel), excluding the debug liveness passes and DWARF emission. This most significantly affects MachineSink, which now needs to consider all used registers of a debug value when sinking, but for most passes this change is simply replacing getDebugOperand(0) with an iteration over all debug operands. Differential Revision: https://reviews.llvm.org/D92578
2021-02-17[CodeGen] Use range-based for loops (NFC)Kazu Hirata1-5/+2
2021-02-17[MachineSink] Add a loop sink limitSjoerd Meijer1-0/+17
To make sure compile-times don't regress, add an option to restrict the number of instructions considered for sinking as alias analysis can be expensive and for the same reason also skip large blocks. Differential Revision: https://reviews.llvm.org/D96485
2021-01-27[MachineLICM][MachineSink] Move SinkIntoLoop to MachineSink.Sjoerd Meijer1-0/+156
This moves SinkIntoLoop from MachineLICM to MachineSink. The motivation for this work is that hoisting is a canonicalisation transformation, but we do not really have a good story to sink instructions back if that is better, e.g. to reduce live-ranges, register pressure and spilling. This has been discussed a few times on the list, the latest thread is: https://lists.llvm.org/pipermail/llvm-dev/2020-December/147184.html There it was pointed out that we have the LoopSink IR pass, but that works on IR, lacks register pressure informatiom, and is focused on profile guided optimisations, and then we have MachineLICM and MachineSink that both perform sinking. MachineLICM is more about hoisting and CSE'ing of hoisted instructions. It also contained a very incomplete and disabled-by-default SinkIntoLoop feature, which we now move to MachineSink. Getting loop-sinking to do something useful is going to be at least a 3-step approach: 1) This is just moving the code and is almost a NFC, but contains a bug fix. This uses helper function `isLoopInvariant` that was factored out in D94082 and added to MachineLoop. 2) A first functional change to make loop-sink a little bit less restrictive, which it really is at the moment, is the change in D94308. This lets it do more (alias) analysis using functions in MachineSink, making it a bit more powerful. Nothing changes much: still off by default. But it shows that MachineSink is a better home for this, and it starts using its functionality like `hasStoreBetween`, and in the next step we can use `isProfitableToSinkTo`. 3) This is the going to be he interesting step: decision making when and how many instructions to sink. This will be driven by the register pressure, and deciding if reducing live-ranges and loop sinking will help in better performance. 4) Once we are happy with 3), this should be enabled by default, that should be the end goal of this exercise. Differential Revision: https://reviews.llvm.org/D93694
2021-01-08[Target, Transforms] Use *Set::contains (NFC)Kazu Hirata1-1/+1
2021-01-05[MachineSink] Construct SmallVector with iterator ranges (NFC)Kazu Hirata1-3/+2
2021-01-04[llvm] Use llvm::any_of (NFC)Kazu Hirata1-3/+3
2020-12-31[CodeGen] Construct SmallVector with iterator ranges (NFC)Kazu Hirata1-2/+1
2020-12-27[MachineSink] add threshold in machinesink pass to reduce compiling time.Chen Zheng1-0/+36
2020-11-04[MachineSink] add more profitable pattern.Chen Zheng1-7/+72
Add more profitable sinking patterns if the target bb register pressure is not too high. Reviewed By: qcolombet Differential Revision: https://reviews.llvm.org/D88126
2020-11-01[MachineSink] sink more profitable loadsChen Zheng1-2/+82
Reviewed By: qcolombet Differential Revision: https://reviews.llvm.org/D86864
generated by cgit v1.1 at 2025-10-01 00:51:35 +0000