aboutsummaryrefslogtreecommitdiff
path: root/llvm/lib/CodeGen/RegisterCoalescer.cpp
AgeCommit message (Collapse)AuthorFilesLines
2023-12-15Reapply "RegisterCoalescer: Add implicit-def of super register when ↵Matt Arsenault1-9/+42
coalescing SUBREG_TO_REG" This reverts commit 69c4930aad9659ec6ab846c8e7124d6afe044b1e. See if this sticks after a few more coalescer assertions are fixed.
2023-12-13RegisterCoalescer: Fix implicit operand handling during rematerialize (#75271)Matt Arsenault1-2/+1
If the rematerialize was placing a subregister into a super register, and implicit operands referenced the original register, we need to add undef flags to the now-subregister indexed implicit operands. Depends #75152
2023-12-13RegisterCoalescer: Add undef flags in removePartialRedundancy (#75152)Matt Arsenault1-0/+15
If the copy being hoisted was undef, we have the same problems that eliminateUndefCopy needs to solve. We would effectively be introducing a new live out implicit_def. We need to add an undef flag to avoid artificially introducing a live through undef value. Previously, the verifier would fail due to the dead def inside the loop providing the live in value for the %1 use.
2023-12-06Revert "Reapply "RegisterCoalescer: Add implicit-def of super register when ↵Thurston Dang1-42/+9
coalescing SUBREG_TO_REG"" This reverts commit 1f283a60a4bb896fa2d37ce00a3018924be82b9f. Reason: breaks MSan buildbot (https://lab.llvm.org/buildbot/#/builders/74/builds/24077)
2023-12-06Reapply "RegisterCoalescer: Add implicit-def of super register when ↵Matt Arsenault1-9/+42
coalescing SUBREG_TO_REG" This reverts commit 9e50c6e6b5741895f58f3e530004052844b6af9f. A few assertion and verifier errors have been fixed in the coalescer and allocator, so hopefully this sticks this time.
2023-12-04RegisterCoalescer: Fix producing malformed IMPLICIT_DEFs (#73784)Matt Arsenault1-2/+9
If this was coalescing a SUBREG_TO_REG as a copy, the resulting instruction would be an IMPLICIT_DEF with an unexpected 2 immediate operands, which need to be dropped. Until recently the verifier did not catch this error, and an assert would fire if later the broken IMPLICIT_DEF was rematerialized P #73758 is related, it changes the failure mode to a verifier error.
2023-11-07Revert "Reapply "RegisterCoalescer: Add implicit-def of super register when ↵Mitch Phillips1-42/+9
coalescing SUBREG_TO_REG"" This reverts commit ba385ae210b3659bc9dfb78ef1d280d03c2c3b5a. Reason: Broke the MSan buildbot. See comments on https://github.com/llvm/llvm-project/commit/ba385ae210b3659bc9dfb78ef1d280d03c2c3b5a for more information.
2023-11-07RegisterCoalescer: Clear isSSA propertyMatt Arsenault1-0/+5
Fix some expensive checks verifier errors on MIR tests.
2023-11-07Reapply "RegisterCoalescer: Add implicit-def of super register when ↵Matt Arsenault1-9/+42
coalescing SUBREG_TO_REG" This reverts commit e0f86ca2004b2d87ffe3c1e8242650a29fa98a82. This was hitting some assertions which have since been relaxed.
2023-10-31RegisterCoalescer: Relax assert for super register def rematerialization ↵Matt Arsenault1-4/+9
(#69088)
2023-10-05MachineFunctionPass: Clear properties before running function (#67962)Matt Arsenault1-8/+0
This ensures !isSSA checks in the function work if the input MIR happened to appear as SSA.
2023-10-02Revert "RegisterCoalescer: Add implicit-def of super register when ↵Kirill Stoimenov1-42/+9
coalescing SUBREG_TO_REG" This reverts commit 414ff812d6241b728754ce562081419e7fc091eb.
2023-10-02Fix MSVC "cannot convert from 'llvm::Register' to 'llvm::MCRegister'" build ↵Simon Pilgrim1-2/+3
error. NFCI.
2023-10-02RegisterCoalescer: Add implicit-def of super register when coalescing ↵Matt Arsenault1-9/+42
SUBREG_TO_REG Currently coalescing with SUBREG_TO_REG introduces an invisible load bearing undef. There is liveness for the super register not represented in the MIR. This is part 1 of a fix for regressions that appeared after b7836d856206ec39509d42529f958c920368166b. The allocator started recognizing undef-def subregister MOVs as copies. Since there was no representation for the dependency on the high bits, different undef segments of the super register ended up disconnected and downstream users ended up observing different undefs than they did previously. This does not yet fix the regression. The isCopyInstr handling needs to start handling implicit-defs on any instruction. I wanted to include an end to end IR test since the actual failure only appeared with an interaction between the coalescer and the allocator. It's a bit bigger than I'd like but I'm having a bit of trouble reducing it to something which definitely shows a diff that's meaningful. The same problem likely exists everywhere trying to do anything with SUBREG_TO_REG. I don't understand how this managed to be broken for so long. This needs to be applied to the release branch. https://reviews.llvm.org/D156345
2023-10-02[CodeGen] Fix -Wunused-variable in RegisterCoalescer.cpp (NFC)Jie Fu1-1/+1
/llvm-project/llvm/lib/CodeGen/RegisterCoalescer.cpp:1320:18: error: unused variable 'DefSubIdx' [-Werror,-Wunused-variable] const unsigned DefSubIdx = DefMI->getOperand(0).getSubReg(); ^ 1 error generated.
2023-10-02RegisterCoalescer: Avoid redundant implicit-def on rematerializeMatt Arsenault1-2/+11
If this was coalescing a def of a subregister with a def of the super register, it was introducing a redundant super-register def and marking the subregister def as dead. Resulting in something like: dead $eax = MOVr0, implicit-def $rax, implicit-def $rax Avoid this by checking if the new instruction already has the super def, so we end up with this instead: dead $eax = MOVr0, implicit-def $rax The dead flag looks suspicious to me, seems like it's easy to buggily interpret dead def of subreg and a non-dead def of an aliasing register. It seems to be intentional though. https://reviews.llvm.org/D156343
2023-10-02RegisterCoalescer: Handle implicit-def of a super register when rematerializingMatt Arsenault1-3/+29
Permit an implicit-def of a virtual register when rematerializing if it defines a super register of a subregister def. The rematerialization pre-legality check should really have been checking the implicit operands, but that should be fixed separately. https://reviews.llvm.org/D156331
2023-10-02RegisterCoalescer: Forcibly leave SSA to avoid MIR test errorsMatt Arsenault1-0/+8
Not sure how to produce a test that demonstrates the problem today. The coalescer would have to introduce a verifier caught SSA violation, like multiple defs of a virtual register. I'm not sure what would do that now, but an upcoming patch will. https://reviews.llvm.org/D156271
2023-09-12RegisterCoalescer: Don't delete IMPLICIT_DEF if it's live into the same blockMatt Arsenault1-1/+6
Live out implicit_defs need to be kept, but the check for this only checked if the block parent was the same. This doesn't work if the parent blocks are the same but the value is live. Fixes verifier error "Instruction ending live segment doesn't read the register", which would appear at the coalesced non-implicit_def def. Fixes #38788 https://reviews.llvm.org/D158882
2023-09-12RegisterCoalescer: Correctly set valid lanes when keeping live out implicit defsMatt Arsenault1-17/+14
This fixes some verifier errors when live out implicit defs are coalesced with identity copies. Fixes some reduced testcases from issue #38788 but doesn't solve the original failure. I was surprised this seems to obviate the special casing in analyzeValue that's been there since the subregister liveness support went in. https://reviews.llvm.org/D158850
2023-07-31RegisterCoalescer: Remove dubious dropping of implicit virtual register defsMatt Arsenault1-3/+2
Don't understand why this would either be OK or necessary, but doesn't appear to happen in any tests. This was introduced way back in 76e66c31a0481e72d1ff86c56028d850b6c33cff https://reviews.llvm.org/D156265
2023-07-24RegisterCoaleser: Fix empty subrange verifier errorMatt Arsenault1-2/+9
In this example an implicit def had live-out undef subrange defs. After coalescing with the def from a previous block, the undef-defed lanes are no longer live out of the block in the new interval. An empty subrange was tenatively created for these lanes, but it must be deleted.
2023-07-24RegisterCoalescer: Fix verifier error on redef of subregister for live out ↵Matt Arsenault1-1/+1
implicit_defs A live out implicit_def wasn't deleted, but the subranges weren't correctly updated. The main range was correct but the def corresponding to the initial main range def instruction was missing from the lanes redefined in another block. The written lanes are not quite the same as the valid lanes in the case of an implicit_def. Fixes verifier error in blender. There is an additional verifier in some of the testcase variants where an empty subrange remains.
2023-06-21RegisterCoalescer: Fix name of passMatt Arsenault1-5/+5
I finally snapped and fixed this inconsistency.
2023-06-16[MC] Add MCRegisterInfo::regunits for iteration over register unitsSergei Barannikov1-11/+10
Reviewed By: foad Differential Revision: https://reviews.llvm.org/D152098
2023-06-01[CodeGen] Make use of MachineInstr::all_defs and all_uses. NFCI.Jay Foad1-8/+8
Differential Revision: https://reviews.llvm.org/D151424
2023-05-24Bump coalescing limitMatthias Braun1-1/+1
This bumps the "large-interval-freq-threshold" limit in the register coalescer to 256. The limit was introduced in https://reviews.llvm.org/D59143 without much justify for the particular value "100", so I hope bumping it is ok. This change is motivated by bad codegen for the popular crc32c algorithm; the code is often based/copied from this implementation: https://github.com/htot/crc32c/blob/master/crc32c/crc32intelc.cc which uses a duffs-device pattern with 128 switch-cases. There are examples in RocksDB (https://github.com/facebook/rocksdb/blob/main/util/crc32c.cc) and Folly (https://github.com/facebook/folly/blob/main/folly/hash/detail/Crc32cDetail.cpp) which are important use cases for us. Differential Revision: https://reviews.llvm.org/D150994
2023-05-24[RegisterCoalescer] Fix updating LiveIntervals in joinReservedPhysRegJay Foad1-2/+3
Live intervals for physical registers are calculated lazily on demand. In a case like this: 16B %0:gpr32 = IMPLICIT_DEF 32B $wzr = COPY %0 if the live interval for $wzr did not already exist then the update code in joinReservedPhysReg would create it with a definition at 32B, which would remain even after the COPY was deleted. Differential Revision: https://reviews.llvm.org/D151314
2023-05-18[RegisterCoalescer] Remove DbgMergedVRegNums (NFC)Heejin Ahn1-18/+2
Not sure what this was originally intended for, but this seems to be unused. It didn't seem to be used when it was first added in D64630 either. Reviewed By: jmorse Differential Revision: https://reviews.llvm.org/D150606
2023-05-06[RegisterCoalescer] Fix problem with IMPLICIT_DEF live-in to an invokeJay Foad1-2/+11
Give up on erasing an IMPLICIT_DEF if it might be live-in to a call instruction in a basic block with EH pad successors. This fixes a liveness bug that will be diagnosed by MachineVerifer when D149947 lands. Differential Revision: https://reviews.llvm.org/D149954
2023-01-13[CodeGen] Remove uses of Register::isPhysicalRegister/isVirtualRegister. NFCCraig Topper1-24/+18
Use isPhysical/isVirtual methods. Reviewed By: foad Differential Revision: https://reviews.llvm.org/D141715
2022-09-23[RegisterCoalescer] fix dst subreg replacement during remat copy trickAfanasyev Ivan1-1/+11
Instructions might use definition register as its "undef" operand. It happens on architectures with predicated executon: ``` %0:subreg = instruction op_1, ..., op_N, undef %0:subreg, op_N+2, ... ``` RegisterCoalescer should take into account all remat instruction operands during destination subregister fixup. ``` ; remat result before fix: %1 = instruction op_1, ..., op_N, undef %1:subreg, op_N+2, ... ; remat result after fix (correct): %1 = instruction op_1, ..., op_N, undef %1, op_N+2, ... ``` Differential Revision: https://reviews.llvm.org/D125657
2022-09-21[RegisterCoalescer] Use LiveRangeEdit to handle rematerializationGuozhi Wei1-20/+16
This patch uses the API provided by LiveRangeEdit to handle rematerialization. It will make future maintenance and improvement more easier. No functional change. Differential Revision: https://reviews.llvm.org/D133610
2022-09-13RegisterCoalescer: Fix verifier error when merging copy of undefMatt Arsenault1-11/+13
There's no real read of the register, so the copy introduced a new live value. Make sure we introduce a replacement implicit_def instead of just erasing the copy. Found from llvm-reduce since it tries to set undef on everything.
2022-09-06[RegisterCoalescer] Fix crash on early clobbered subreg operands.Daniil Fukalov1-2/+4
The issue was with processing two subregs of the same reg are used in the same instruction (e.g. inline asm): "def early-clobber" and other just "def". Register coalescer ran in bad recursion if the early clobbered subreg is second in the following sequence of COPYs. Reviewed By: arsenm Differential Revision: https://reviews.llvm.org/D127136
2022-08-23[NFC] Fix typo in dbg message in RegisterCoalescer.Thomas Symalla1-1/+1
funcion => function
2022-08-08[llvm] LLVM_FALLTHROUGH => [[fallthrough]]. NFCFangrui Song1-2/+2
With C++17 there is no Clang pedantic warning or MSVC C5051.
2022-08-07[llvm] LLVM_NODISCARD => [[nodiscard]]. NFCFangrui Song1-18/+18
With C++17 there is no Clang pedantic warning.
2022-07-29RegisterCoalescer: Shrink main range after shrinking subrangesMatt Arsenault1-0/+1
If the subregister uses were dead, this would leave the main range segment pointing to a deleted instruction. Not sure if this should try to avoid shrinking if we know we don't have dead components.
2022-07-18CodeGen: Remove AliasAnalysis from regallocMatt Arsenault1-1/+1
This was stored in LiveIntervals, but not actually used for anything related to LiveIntervals. It was only used in one check for if a load instruction is rematerializable. I also don't think this was entirely correct, since it was implicitly assuming constant loads are also dereferenceable. Remove this and rely only on the invariant+dereferenceable flags in the memory operand. Set the flag based on the AA query upfront. This should have the same net benefit, but has the possible disadvantage of making this AA query nonlazy. Preserve the behavior of assuming pointsToConstantMemory implying dereferenceable for now, but maybe this should be changed.
2022-07-17[CodeGen] Qualify auto variables in for loops (NFC)Kazu Hirata1-1/+1
2022-03-16[NFC][CodeGen] Rename some functions in MachineInstr.h and remove duplicated ↵Shengchen Kan1-1/+1
comments
2021-12-06[CodeGen] Use range-based for loops (NFC)Kazu Hirata1-4/+4
2021-11-23[llvm] Use range-based for loops (NFC)Kazu Hirata1-6/+6
2021-11-10[llvm] Use make_early_inc_range (NFC)Kazu Hirata1-6/+2
2021-11-01[CodeGen] Use make_early_inc_range (NFC)Kazu Hirata1-3/+2
2021-09-23[CodeGen] RegisterCoalescer::buildVRegToDbgValueMap - use const-ref value in ↵Simon Pilgrim1-1/+1
for-range loop. NFCI. Avoid unnecessary copies, reported by MSVC static analyzer.
2021-07-22[RegisterCoalescer] Make resolveConflicts aware of earlyclobberShihPo Hung1-2/+4
Prior to this patch, it skipped the instruction defining VNI when checking if the tainted lanes are used. In the given example, VRGATHER is an illegal instruction because its DstReg overlaps with SrcReg. Therefore we need to check the defining instruction as well when there is an earlyclobber constraint. Reviewed By: qcolombet Differential Revision: https://reviews.llvm.org/D105684
2021-07-21Prevent dead uses in register coalescer after rematerializationStanislav Mekhanoshin1-0/+16
The coalescer does not check if register uses are available at the point of rematerialization. If it attempts to rematerialize an instruction with such uses it can end up with use without a def. LiveRangeEdit does such check during rematerialization, so just call LiveRangeEdit::allUsesAvailableAt() to avoid the problem. Differential Revision: https://reviews.llvm.org/D106396
2021-07-14[RegisterCoalescer] Resolve conflict based on liveness of subregisterRuiling Song1-3/+33
Currently we are resolving lane/subregister conflict by visiting instructions sequentially in current block to see whether there is any use of the tainted lanes. To save compile time, we are not doing further check in successor blocks. This sounds reasonable without subgregister liveness. But since we have added subregister liveness tracking capability to register coalescer, we can easily determine whether we have subregister liveness conflict by checking subranges. This would help coalescing more COPYs for target that enables subregister liveness tracking. Reviewed by: arsenm, qcolombet Differential Revision: https://reviews.llvm.org/D104509
generated by cgit v1.1 at 2025-10-19 04:52:11 +0000