aboutsummaryrefslogtreecommitdiff
path: root/llvm/lib/CodeGen/InlineSpiller.cpp
AgeCommit message (Collapse)AuthorFilesLines
2022-05-16[LiveIntervals] Add range accessors for value numbers [nfc]Philip Reames1-1/+1
2022-03-16[NFC][CodeGen] Rename some functions in MachineInstr.h and remove duplicated ↵Shengchen Kan1-2/+2
comments
2022-03-16Cleanup codegen includesserge-sans-paille1-1/+0
This is a (fixed) recommit of https://reviews.llvm.org/D121169 after: 1061034926 before: 1063332844 Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup Differential Revision: https://reviews.llvm.org/D121681
2022-03-10Revert "Cleanup codegen includes"Nico Weber1-0/+1
This reverts commit 7f230feeeac8a67b335f52bd2e900a05c6098f20. Breaks CodeGenCUDA/link-device-bitcode.cu in check-clang, and many LLVM tests, see comments on https://reviews.llvm.org/D121169
2022-03-10Cleanup codegen includesserge-sans-paille1-1/+0
after: 1061034926 before: 1063332844 Differential Revision: https://reviews.llvm.org/D121169
2021-12-05[llvm] Use range-based for loops (NFC)Kazu Hirata1-3/+1
2021-11-28[llvm] Use range-based for loops (NFC)Kazu Hirata1-3/+1
2021-11-25[llvm] Use range-based for loops (NFC)Kazu Hirata1-3/+1
2021-11-08[llvm] Use make_early_inc_range (NFC)Kazu Hirata1-36/+24
2021-11-05[llvm] Use make_early_inc_range (NFC)Kazu Hirata1-3/+2
2021-10-25[DebugInfo][InstrRef] Track values fused into stack spillsJeremy Morse1-0/+33
During register allocation, some instructions can have stack spills fused into them. It means that when vregs are allocated on the stack we can convert: SETCCr %0 DBG_VALUE %0 to SETCCm %stack.0 DBG_VALUE %stack.0 Unfortunately instruction referencing finds this harder: a store to the stack doesn't have a specific operand number, therefore we don't substitute the old operand for a new operand, and the location is dropped. This patch implements a solution: just recognise the memory operand attached to an instruction with a Special Number (TM), and record a substitution between the old value and the new one. This patch adds substitution code to InlineSpiller to record such fused spills, and tracking in InstrRefBasedLDV to recognise such values, and produce the value numbers for them. Everything to do with the movement of stack-defined values is already handled in InstrRefBasedLDV. Differential Revision: https://reviews.llvm.org/D111317
2021-05-05[GreedyRA] Add support for invoke statepoint with tied-defs.Serguei Katkov1-0/+8
statepoint instruction uses tied-def registers to represent live gc value which is use and def at the same time on a call. At the same time invoke statepoint instruction is a last split point which can throw and jump to landing pad. As a result we have instructon which is last split point with tied-defs registers and we need to teach Greedy RA to work with it. The option -use-registers-for-gc-values-in-landing-pad controls whether statepoint lowering will generate tied-defs for invoke statepoint and is off by default now. To resolve all issues the following changes has been done. 1) Last Split point for invoke statepoint should be statepoint itself If statepoint has a def it is a relocated gc pointer and it should be available in landing pad. So we cannot split interval after statepoint at end of basic block. 2) Do not split interval on tied-def If end of interval for overlap utility is a use which has tied-def we should not split interval on this instruction due to in this case use and def may have different registers and it breaks tied-def property. 3) Take into account Last Split Point for enterIntvAtEnd If the use after Last Split Point is a def so it should be tied-def and we can take the def of the tied-use as ParentVNI and thus tied-use and tied-def will be live in resulting interval. 4) Handle the case when def is after LIP in InlineSpiller If def of LI is after last insertion point of basic block we cannot hoist in this BB. The example of such instruction is invoke statepoint where def represents the relocated live gc pointer. Invoke is a last insertion point and its def is located after it. In this case there is no place to insert spill and we bail out. 5) Fix removeBackCopies to account empty copies RegAssignMap cannot hold empty interval, so do not set stop to kill value if it produces empty interval. This can happen if we remove back-copy and right before that we have another back-copy. For example, for parent %0 we can get %1 = COPY %0 %2 = COPY %0 while we removing %2 we cannot set kill for %1 due to its empty. 6) Do not hoist copy to BB if its def is after LSP If the parent def is a LastSplitPoint or later we cannot hoist copy to this basic block because inserted copy (or re-materialization) will be located before the def. All parts have been reviewed separately as follows: https://reviews.llvm.org/D100747 https://reviews.llvm.org/D100748 https://reviews.llvm.org/D100750 https://reviews.llvm.org/D100927 https://reviews.llvm.org/D100945 https://reviews.llvm.org/D101028 Reviewers: reames, rnk, void, MatzeB, wmi, qcolombet Reviewed By: reames, qcolombet Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D101150
2021-04-23[InlineSpiller] Clean-up isSpillCandBBSerguei Katkov1-6/+1
This is mostly NFC except that for end of BB not previous slot is used. Idx is used to find a def of sibling live interval in that slot. The def on end of MBB and on previous slot of end MBB should be the same, so it should be NFC. Reviewers: reames, qcolombet, MatzeB, wmi, rnk Reviewed By: rnk Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D100922
2021-03-10[DebugInfo] Handle DBG_VALUES with multiple variable location operands in MIRStephen Tozer1-1/+1
This patch adds handling for DBG_VALUE_LIST in the MIR-passes (after finalize-isel), excluding the debug liveness passes and DWARF emission. This most significantly affects MachineSink, which now needs to consider all used registers of a debug value when sinking, but for most passes this change is simply replacing getDebugOperand(0) with an iteration over all debug operands. Differential Revision: https://reviews.llvm.org/D92578
2021-03-04Revert "[DebugInfo] Add new instruction and DIExpression operator for ↵Stephen Tozer1-1/+1
variadic debug values" This reverts commit d07f106f4a48b6e941266525b6f7177834d7b74e.
2021-03-04[DebugInfo] Add new instruction and DIExpression operator for variadic debug ↵gbtozers1-1/+1
values This patch adds a new instruction that can represent variadic debug values, DBG_VALUE_VAR. This patch alone covers the addition of the instruction and a set of basic code changes in MachineInstr and a few adjacent areas, but does not correctly handle variadic debug values outside of these areas, nor does it generate them at any point. The new instruction is similar to the existing DBG_VALUE instruction, with the following differences: the operands are in a different order, any number of values may be used in the instruction following the Variable and Expression operands (these are referred to in code as “debug operands”) and are indexed from 0 so that getDebugOperand(X) == getOperand(X+2), and the Expression in a DBG_VALUE_VAR must use the DW_OP_LLVM_arg operator to pass arguments into the expression. The new DW_OP_LLVM_arg operator is only valid in expressions appearing in a DBG_VALUE_VAR; it takes a single argument and pushes the debug operand at the index given by the argument onto the Expression stack. For example the sub-expression `DW_OP_LLVM_arg, 0` has the meaning “Push the debug operand at index 0 onto the expression stack.” Differential Revision: https://reviews.llvm.org/D82363
2021-02-19[NFC][Regalloc] Share the VirtRegAuxInfo object with LiveRangeEditMircea Trofin1-15/+19
VirtRegAuxInfo is an extensibility point, so the register allocator's decision on which implementation to use should be communicated to the other users - namely, LiveRangeEdit. Differential Revision: https://reviews.llvm.org/D96898
2021-02-17[NFC][RegAlloc] InlineSpiller::Original is a RegisterMircea Trofin1-1/+1
2021-01-19[X86] Fix tile spill merge issue.Luo, Yuanke1-1/+1
This is a additional bug fix for c5be0e0cc0. The distance for the spill instructions is wrong in previous patch. Differential Revision: https://reviews.llvm.org/D94772
2021-01-13[InlineSpiller] Re-tie operands if folding failedSerguei Katkov1-5/+22
InlineSpiller::foldMemoryOperand unties registers before an attempt to fold and does not restore tied-ness in case of failure. I do not have a particular test for demo of invalid behavior. This is something of clean-up. It is better to keep the behavior correct in case some time in future it happens. Reviewers: reames, dantrushin Reviewed By: dantrushin, reames Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D94389
2021-01-11Fix minor build issue (NFC)David Stuttard1-5/+5
Change [x86] Fix tile register spill issue was causing problems for our build using gcc-5.4.1 The problem was caused by this line: for (const MachineInstr &MI : make_range(MIS.begin(), MI)) where MI was previously defined as a MachineBasicBlock iterator. Differential Revision: https://reviews.llvm.org/D94415
2021-01-11[X86] Fix tile register spill issue.Luo, Yuanke1-5/+33
The tile register spill need 2 instructions. %46:gr64_nosp = MOV64ri 64 TILESTORED %stack.2, 1, killed %46:gr64_nosp, 0, $noreg, %43:tile The first instruction load the stride to a GPR, and the second instruction store tile register to stack slot. The optimization of merge spill instruction is done after register allocation. And spill tile register need create a new virtual register to for stride, so we can't hoist tile spill instruction in postOptimization() of register allocation. We can't hoist TILESTORED alone and we can't hoist the 2 instuctions together because MOV64ri will clobber some GPR. This patch is to disble the spill merge for any spill which need 2 instructions. Differential Revision: https://reviews.llvm.org/D93898
2020-12-10[X86] AMX programming model.Luo, Yuanke1-0/+2
This patch implements amx programming model that discussed in llvm-dev (http://lists.llvm.org/pipermail/llvm-dev/2020-August/144302.html). Thank Hal for the good suggestion in the RA. The fast RA is not in the patch yet. This patch implemeted 7 components. 1. The c interface to end user. 2. The AMX intrinsics in LLVM IR. 3. Transform load/store <256 x i32> to AMX intrinsics or split the type into two <128 x i32>. 4. The Lowering from AMX intrinsics to AMX pseudo instruction. 5. Insert psuedo ldtilecfg and build the def-use between ldtilecfg to amx intruction. 6. The register allocation for tile register. 7. Morph AMX pseudo instruction to AMX real instruction. Change-Id: I935e1080916ffcb72af54c2c83faa8b2e97d5cb0 Differential Revision: https://reviews.llvm.org/D87981
2020-11-03[NFC] Use [MC]Register in register allocationGaurav Jain1-2/+2
Differential Revision: https://reviews.llvm.org/D90725
2020-10-08[NFC][MC] MCRegister API typing.Mircea Trofin1-1/+1
Mostly LiveIntervals, with their effects (users). Differential Revision: https://reviews.llvm.org/D89018
2020-09-16[NFC][Regalloc] accessors for 'reg' and 'weight'Mircea Trofin1-12/+13
Also renamed the fields to follow style guidelines. Accessors help with readability - weight mutation, in particular, is easier to follow this way. Differential Revision: https://reviews.llvm.org/D87725
2020-08-05[Statepoints] Operand folding in presense of tied registers.Denis Antrushin1-0/+11
Implement proper folding of statepoint meta operands (deopt and GC) when statepoint uses tied registers. For deopt operands it is just about properly preserving tiedness in new instruction. For tied GC operands folding is a little bit more tricky. We can fold tied GC operands only from InlineSpiller, because it knows how to properly reload tied def after it was turned into memory operand. Other users (e.g. peephole) cannot properly fold such operands as they do not know how (or when) to reload them from memory. We do this by un-tieing operand we want to fold in InlineSpiller and allowing to fold only untied operands in foldPatchpoint.
2020-07-06DomTree: Remove getChildren() accessorNicolai Hähnle1-8/+2
Summary: Avoid exposing details about how children are stored. This will enable subsequent type-erasure changes. New methods are introduced to cover common access patterns. Change-Id: Idb5f4b1b9c84e4cc71ddb39bb52a388682f5674f Reviewers: arsenm, RKSimon, mehdi_amini, courbet Subscribers: qcolombet, sdardis, wdng, hiraditya, jrtc27, zzheng, atanasyan, asbirlea, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D83083
2020-06-30[NFC] fix diagnosticJF Bastien1-1/+1
It's pretty silly to diagnose on a scalar copy but the build does that: loop variable 'SibReg' of type 'const llvm::Register' creates a copy from type 'const llvm::Register' [-Wrange-loop-analysis]
2020-06-30RegAlloc: Start using RegisterMatt Arsenault1-46/+46
2020-06-17[InlineSpiller] add assert about spills post terminatorsNick Desaulniers1-0/+3
Summary: This invariant is being violated in the test case https://reviews.llvm.org/D77849, related to the use of the relatively new ability for callbr to have return values, and MachineBasicBlocks with INLINEASM_BR terminators to emit live out register defs. As noted in the comment, this triggers invariant violations in MachineVerifier via `llc -verify-machineinstrs` or `llc -verify-regalloc`, since only MachineInstrs that are terminators are allowed to follow the first terminator. https://reviews.llvm.org/D75098 may rework this very assertion if we're spilling via a (proposed) TCOPY MachineInstr. Reviewers: void, efriedma, arsenm Reviewed By: efriedma Subscribers: qcolombet, wdng, hiraditya, llvm-commits, srhines Tags: #llvm Differential Revision: https://reviews.llvm.org/D78166
2020-04-21[InlineSpiller] simplify insertReload() NFCNick Desaulniers1-14/+15
Summary: The repeated use of std::next() on a MachineBasicBlock::iterator was clever, but we only need to reconstruct the iterator post creation of the spill instruction. This helps simplifying where we plan to place the spill, as discussed in D77849. From here, we can simplify the code a little by flipping the return code of a helper. Reviewers: efriedma Reviewed By: efriedma Subscribers: qcolombet, hiraditya, llvm-commits, srhines Tags: #llvm Differential Revision: https://reviews.llvm.org/D78520
2020-04-10Split LiveRangeCalc in LiveRangeCalc/LiveIntervalCalc. NFCMarcello Maggioni1-1/+1
Summary: Refactor LiveRangeCalc such that it is now split into two classes The objective is to split all the "register specific" logic away from LiveRangeCalc. The two new classes created are: - LiveRangeCalc - is meant as a generic class to compute and modify live ranges in a generic way. This class should deal only with SlotIndices and VNInfo objects. - LiveIntervalCals - is meant to be equivalent to the old LiveRangeCalc. It computes the liveness virtual registers tracked by a LiveInterval object. With this refactoring LiveRangeCalc can be used to implement tracking of liveness of LiveRanges that represent other things than just registers. Subscribers: MatzeB, qcolombet, mgorny, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D76584
2020-03-09Move Spiller.h from lib/ directory path to include/CodeGen. NFCMarcello Maggioni1-1/+1
This allows Spiller.h to be used and included outside of the lib/CodeGen directory. For example to be used in the lib/Target directory or other places.
2020-03-02[InlineSpiller] Relax re-materialization restriction for statepointSerguei Katkov1-1/+14
We should be careful to allow count of re-materialization of operands to be less then number of physical registers. STATEPOINT instruction has a variable number of operands and potentially very big. So re-materialization for all operands is disabled at the moment if restrict-statepoint-remat is true. The patch relaxes the re-materialization restriction for STATEPOINT instruction allowing it for fixed operands. Specifically it is about call target. Reviewers: reames Reviewed By: reames Subscribers: llvm-commits, qcolombet, hiraditya Differential Revision: https://reviews.llvm.org/D75335
2020-02-10[CSInfo] Use isCandidateForCallSiteEntry() when updating the CSInfoDjordje Todorovic1-1/+2
Use the isCandidateForCallSiteEntry(). This should mostly be an NFC, but there are some parts ensuring the moveCallSiteInfo() and copyCallSiteInfo() operate with call site entry candidates (both Src and Dest should be the call site entry candidates). Differential Revision: https://reviews.llvm.org/D74122
2020-01-03Fix typo "psuedo" in commentsJay Foad1-1/+1
2020-01-01[NFC] Fixes -Wrange-loop-analysis warningsMark de Wever1-3/+3
This avoids new warnings due to D68912 adds -Wrange-loop-analysis to -Wall. Differential Revision: https://reviews.llvm.org/D71857
2019-12-02[MIBundles] Move analyzePhysReg out of MIBundleOperands iterator (NFC).Florian Hahn1-2/+1
analyzePhysReg does not really fit into the iterator and moving it makes it easier to change the base iterator. Reviewers: evandro, t.p.northover, paquette, MatzeB, arsenm, qcolombet Reviewed By: arsenm Differential Revision: https://reviews.llvm.org/D70559
2019-12-02[MIBundles] Move analyzeVirtReg out of MIBundleOperands iterator (NFC).Florian Hahn1-5/+3
analyzeVirtReg does not really fit into the iterator and moving it makes it easier to change the base iterator. Reviewers: evandro, t.p.northover, paquette, MatzeB, arsenm, qcolombet Reviewed By: qcolombet Differential Revision: https://reviews.llvm.org/D70558
2019-10-17Move LiveRangeCalc header to publicily available position. NFCMarcello Maggioni1-1/+1
Differential Revision: https://reviews.llvm.org/D69078 llvm-svn: 375075
2019-10-08[DebugInfo][If-Converter] Update call site info during the optimizationNikola Prica1-1/+1
During the If-Converter optimization pay attention when copying or deleting call instructions in order to keep call site information in valid state. Reviewers: aprantl, vsk, efriedma Reviewed By: vsk, efriedma Differential Revision: https://reviews.llvm.org/D66955 llvm-svn: 374068
2019-08-15Apply llvm-prefer-register-over-unsigned from clang-tidy to LLVMDaniel Sanders1-2/+2
Summary: This clang-tidy check is looking for unsigned integer variables whose initializer starts with an implicit cast from llvm::Register and changes the type of the variable to llvm::Register (dropping the llvm:: where possible). Partial reverts in: X86FrameLowering.cpp - Some functions return unsigned and arguably should be MCRegister X86FixupLEAs.cpp - Some functions return unsigned and arguably should be MCRegister X86FrameLowering.cpp - Some functions return unsigned and arguably should be MCRegister HexagonBitSimplify.cpp - Function takes BitTracker::RegisterRef which appears to be unsigned& MachineVerifier.cpp - Ambiguous operator==() given MCRegister and const Register PPCFastISel.cpp - No Register::operator-=() PeepholeOptimizer.cpp - TargetInstrInfo::optimizeLoadInstr() takes an unsigned& MachineTraceMetrics.cpp - MachineTraceMetrics lacks a suitable constructor Manual fixups in: ARMFastISel.cpp - ARMEmitLoad() now takes a Register& instead of unsigned& HexagonSplitDouble.cpp - Ternary operator was ambiguous between unsigned/Register HexagonConstExtenders.cpp - Has a local class named Register, used llvm::Register instead of Register. PPCFastISel.cpp - PPCEmitLoad() now takes a Register& instead of unsigned& Depends on D65919 Reviewers: arsenm, bogner, craig.topper, RKSimon Reviewed By: arsenm Subscribers: RKSimon, craig.topper, lenary, aemerson, wuzish, jholewinski, MatzeB, qcolombet, dschuff, jyknight, dylanmckay, sdardis, nemanjai, jvesely, wdng, nhaehnle, sbc100, jgravelle-google, kristof.beyls, hiraditya, aheejin, kbarton, fedor.sergeev, javed.absar, asb, rbar, johnrusso, simoncook, apazos, sabuasal, niosHD, jrtc27, MaskRay, zzheng, edward-jones, atanasyan, rogfer01, MartinMosbeck, brucehoult, the_o, tpr, PkmX, jocewei, jsji, Petar.Avramovic, asbirlea, Jim, s.egerton, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D65962 llvm-svn: 369041
2019-08-15[llvm] Migrate llvm::make_unique to std::make_uniqueJonas Devlieghere1-1/+1
Now that we've moved to C++14, we no longer need the llvm::make_unique implementation from STLExtras.h. This patch is a mechanical replacement of (hopefully) all the llvm::make_unique instances across the monorepo. llvm-svn: 369013
2019-08-01Finish moving TargetRegisterInfo::isVirtualRegister() and friends to ↵Daniel Sanders1-7/+5
llvm::Register as started by r367614. NFC llvm-svn: 367633
2019-07-05[CodeGen] Enhance `MachineInstrSpan` to allow the end of MBB to be used.Michael Liao1-3/+3
Summary: - Explicitly specify the parent MBB to allow the end iterator to be used. Reviewers: aprantl, MatzeB, craig.topper, qcolombet Subscribers: arsenm, jvesely, nhaehnle, hiraditya, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D64261 llvm-svn: 365240
2019-06-27[Backend] Keep call site info valid through the backendDjordje Todorovic1-0/+2
Handle call instruction replacements and deletions in order to preserve valid state of the call site info of the MachineFunction. NOTE: If the call site info is enabled for a new target, the assertion from the MachineFunction::DeleteMachineInstr() should help to locate places where the updateCallSiteInfo() should be called in order to preserve valid state of the call site info. ([10/13] Introduce the debug entry values.) Co-authored-by: Ananth Sowda <asowda@cisco.com> Co-authored-by: Nikola Prica <nikola.prica@rt-rk.com> Co-authored-by: Ivan Baev <ibaev@cisco.com> Differential Revision: https://reviews.llvm.org/D61062 llvm-svn: 364536
2019-06-08[SystemZ, RegAlloc] Favor 3-address instructions during instruction selection.Jonas Paulsson1-1/+1
This patch aims to reduce spilling and register moves by using the 3-address versions of instructions per default instead of the 2-address equivalent ones. It seems that both spilling and register moves are improved noticeably generally. Regalloc hints are passed to increase conversions to 2-address instructions which are done in SystemZShortenInst.cpp (after regalloc). Since the SystemZ reg/mem instructions are 2-address (dst and lhs regs are the same), foldMemoryOperandImpl() can no longer trivially fold a spilled source register since the reg/reg instruction is now 3-address. In order to remedy this, new 3-address pseudo memory instructions are used to perform the folding only when the dst and lhs virtual registers are known to be allocated to the same physreg. In order to not let MachineCopyPropagation run and change registers on these transformed instructions (making it 3-address), a new target pass called SystemZPostRewrite.cpp is run just after VirtRegRewriter, that immediately lowers the pseudo to a target instruction. If it would have been possibe to insert a COPY instruction and change a register operand (convert to 2-address) in foldMemoryOperandImpl() while trusting that the caller (e.g. InlineSpiller) would update/repair the involved LiveIntervals, the solution involving pseudo instructions would not have been needed. This is perhaps a potential improvement (see Phabricator post). Common code changes: * A new hook TargetPassConfig::addPostRewrite() is utilized to be able to run a target pass immediately before MachineCopyPropagation. * VirtRegMap is passed as an argument to foldMemoryOperand(). Review: Ulrich Weigand, Quentin Colombet https://reviews.llvm.org/D60888 llvm-svn: 362868
2019-02-12[InlineSpiller] Fix a crash due to lack of forward progress from remat (try 2)Philip Reames1-0/+35
This is a recommit of r335091 Add more test cases for deopt-operands via regalloc, and r335077 [InlineSpiller] Fix a crash due to lack of forward progress from remat specifically for STATEPOINT. They were reverted due to a crash. This change includes the text of both original changes, but also includes three aditional pieces: 1) A bug fix for the observed crash. I had failed to record the failed remat value as live which resulted in an instruction being deleted which still had uses. With the machine verifier, this is caught quickly. Without it, we fail in StackSlotColoring due to an empty live interval from LiveStack. 2) A test case which demonstrates the fix for (1). See @test11. 3) A control flag which defaults to disabling this for the moment. Once I've run more extensive validaton, I will switch the default and then remove this flag. llvm-svn: 353871
2019-01-19Update the file headers across all of the LLVM projects in the monorepoChandler Carruth1-4/+3
to reflect the new license. We understand that people may be surprised that we're moving the header entirely to discuss the new license. We checked this carefully with the Foundation's lawyer and we believe this is the correct approach. Essentially, all code in the project is now made available by the LLVM project under our new license, so you will see that the license headers include that license only. Some of our contributors have contributed code under our old license, and accordingly, we have retained a copy of our old license notice in the top-level files in each project and repository. llvm-svn: 351636
generated by cgit v1.1 at 2025-10-10 11:39:33 +0000