aboutsummaryrefslogtreecommitdiff
path: root/llvm/lib/CodeGen/ModuloSchedule.cpp
AgeCommit message (Collapse)AuthorFilesLines
2024-12-16[NFC] Remove some unnecessary semicolonsDavid Green1-7/+5
All inside LLVM_DEBUG, some of which have been cleaned up by adding block scopes to allow them to format more nicely.
2024-08-26[CodeGen] Use MachineInstr::all_defs (NFC) (#106017)Kazu Hirata1-2/+2
2024-08-15[llvm][CodeGen] Resolve issues when updating live intervals in window ↵Hua Tian1-0/+3
scheduler (#101945) Corrupted live interval information can cause window scheduling to crash in some cases. By adding the missing MBB's live interval information in the ModuloScheduleExpander, the information can be correctly analyzed in the window scheduler.
2024-07-26[CodeGen] Remove AA parameter of isSafeToMove (#100691)Pengcheng Wang1-1/+1
This `AA` parameter is not used and for most uses they just pass a nullptr. The use of `AA` was removed since 8d0383e.
2024-07-10[CodeGen][NewPM] Port `LiveIntervals` to new pass manager (#98118)paperchalice1-3/+3
- Add `LiveIntervalsAnalysis`. - Add `LiveIntervalsPrinterPass`. - Use `LiveIntervalsWrapperPass` in legacy pass manager. - Use `std::unique_ptr` instead of raw pointer for `LICalc`, so destructor and default move constructor can handle it correctly. This would be the last analysis required by `PHIElimination`.
2024-07-09[CodeGen][NewPM] Port `machine-loops` to new pass manager (#97793)paperchalice1-3/+3
- Add `MachineLoopAnalysis`. - Add `MachineLoopPrinterPass`. - Convert to `MachineLoopInfoWrapperPass` in legacy pass manager.
2024-06-12[ModuloSchedule][AArch64] Implement modulo variable expansion for pipelining ↵Yuta Mukai1-0/+640
(#65609) Modulo variable expansion is a technique that resolves overlap of variable lifetimes by unrolling. The existing implementation solves it by making a copy by move instruction for processors with ordinary registers such as Arm and x86. This method may result in a very large number of move instructions, which can cause performance problems. Modulo variable expansion is enabled by specifying -pipeliner-mve-cg. A backend must implement some newly defined interfaces in PipelinerLoopInfo. They were implemented for AArch64. Discourse thread: https://discourse.llvm.org/t/implementing-modulo-variable-expansion-for-machinepipeliner
2024-04-24[CodeGen] Make the parameter TRI required in some functions. (#85968)Xu Zhang1-4/+5
Fixes #82659 There are some functions, such as `findRegisterDefOperandIdx` and `findRegisterDefOperand`, that have too many default parameters. As a result, we have encountered some issues due to the lack of TRI parameters, as shown in issue #82411. Following @RKSimon 's suggestion, this patch refactors 9 functions, including `{reads, kills, defines, modifies}Register`, `registerDefIsDead`, and `findRegister{UseOperandIdx, UseOperand, DefOperandIdx, DefOperand}`, adjusting the order of the TRI parameter and making it required. In addition, all the places that call these functions have also been updated correctly to ensure no additional impact. After this, the caller of these functions should explicitly know whether to pass the `TargetRegisterInfo` or just a `nullptr`.
2024-03-17[CodeGen] Use LocationSize for MMO getSize (#84751)David Green1-2/+2
This is part of #70452 that changes the type used for the external interface of MMO to LocationSize as opposed to uint64_t. This means the constructors take LocationSize, and convert ~UINT64_C(0) to LocationSize::beforeOrAfter(). The getSize methods return a LocationSize. This allows us to be more precise with unknown sizes, not accidentally treating them as unsigned values, and in the future should allow us to add proper scalable vector support but none of that is included in this patch. It should mostly be an NFC. Global ISel is still expected to use the underlying LLT as it needs, and are not expected to see unknown sizes for generic operations. Most of the changes are hopefully fairly mechanical, adding a lot of getValue() calls and protecting them with hasValue() where needed.
2023-06-01[CodeGen] Make use of MachineInstr::all_defs and all_uses. NFCI.Jay Foad1-7/+2
Differential Revision: https://reviews.llvm.org/D151424
2023-01-13[CodeGen] Remove uses of Register::isPhysicalRegister/isVirtualRegister. NFCCraig Topper1-4/+3
Use isPhysical/isVirtual methods. Reviewed By: foad Differential Revision: https://reviews.llvm.org/D141715
2022-12-16[Transforms,CodeGen] std::optional::value => operator*/operator->Fangrui Song1-4/+4
value() has undesired exception checking semantics and calls __throw_bad_optional_access in libc++. Moreover, the API is unavailable without _LIBCPP_NO_EXCEPTIONS on older Mach-O platforms (see _LIBCPP_AVAILABILITY_BAD_OPTIONAL_ACCESS).
2022-12-13[CodeGen] llvm::Optional => std::optionalFangrui Song1-7/+7
2022-12-04[Target] llvm::Optional => std::optionalFangrui Song1-2/+2
The updated functions are mostly internal with a few exceptions (virtual functions in TargetInstrInfo.h, TargetRegisterInfo.h). To minimize changes to LLVMCodeGen, GlobalISel files are skipped. https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2022-10-13Propagate tied operands when copying a MachineInstr.Simon Tatham1-11/+0
MachineInstr's copy constructor works by calling the addOperand method to add each operand of the old MachineInstr to the new one, one by one. But addOperand deliberately avoids trying to replicate ties between operands, on the grounds that the tie refers to operands by index, and the indices aren't necessarily finalized yet. This led to a code generation fault when the machine pipeliner cloned an Arm conditional instruction, and lost the tie between the output register and the input value to be used when the condition failed to execute. Reviewed By: dmgreen Differential Revision: https://reviews.llvm.org/D135434
2022-08-09[MachinePipeliner] Fix Phi generation failure for large stagesYuta Mukai1-21/+54
The previous code overwrites VRMap for prologue stages during Phi generation if a register spans many stages. As a result, the wrong register is used as the one coming from the prologue in Phis at later stages. (A process exists to correct this, but it does not work in all cases.) In addition, VRMap for prologue must be preserved until addBranches(). This patch fixes them by separating the map for Phis into a different variable (VRMapPhi). Reviewed By: bcahoon Differential Revision: https://reviews.llvm.org/D127840
2022-07-13[llvm] Use value instead of getValue (NFC)Kazu Hirata1-4/+4
2022-06-25[llvm] Don't use Optional::hasValue (NFC)Kazu Hirata1-4/+4
This patch replaces Optional::hasValue with the implicit cast to bool in conditionals only.
2022-06-25Revert "Don't use Optional::hasValue (NFC)"Kazu Hirata1-8/+8
This reverts commit aa8feeefd3ac6c78ee8f67bf033976fc7d68bc6d.
2022-06-25Don't use Optional::hasValue (NFC)Kazu Hirata1-8/+8
2022-06-20[llvm] Don't use Optional::getValue (NFC)Kazu Hirata1-1/+1
2022-06-20[llvm] Don't use Optional::hasValue (NFC)Kazu Hirata1-4/+4
2022-06-19[MachinePipeliner] Handle failing constrainRegClassDavid Green1-7/+27
The included test hits a verifier problems as one of the instructions: ``` %113:tgpreven, %114:tgprodd = MVE_VMLSLDAVas16 %12:tgpreven(tied-def 0), %11:tgprodd(tied-def 1), %7:mqpr, %8:mqpr, 0, $noreg, $noreg ``` Has two inputs that come from different PHIs with the same base reg, but conflicting regclasses: ``` %11:tgprodd = PHI %103:gpr, %bb.1, %16:gpr, %bb.2 %12:tgpreven = PHI %103:gpr, %bb.1, %17:gpr, %bb.2 ``` The MachinePipeliner would attempt to use %103 for both the %11 and %12 operands in the prolog, constraining the register class to the common subset of both. Unfortunately there are no registers that are both odd and even, so the second constrainRegClass fails. Fix this situation by inserting a COPY for the second if the call to constrainRegClass fails. The register allocation can then fold that extra copy away. The register allocation of Q regs changed with this test, but the R regs were the same and no new instructions are needed in the final assembly. Differential Revision: https://reviews.llvm.org/D127971
2022-06-07[NFC] Fix spelling/newlines in comments/debug messagesDavid Penry1-1/+1
Just a few spelling mistakes and missing newlines Reviewed By: dmgreen Differential Revision: https://reviews.llvm.org/D127162
2022-06-06[ModuloSchedule] Fix terminator update when peeling.Hendrik Greving1-3/+3
Fixes a bug of us not correctly updating the terminator of the loop's preheader, if multiple terminating branch instructions are present. This is tested through existing tests. The bug itself is hard or not possible to get exposed with the upstream Hexagon backend, because the machine pipeliner checks for an existing preheader, which is defined as a block with only 1 edge into the header. The condition of this bug is a block into the loop with more than 1 edge, and not every downstream target checks for an existing preheader. Differential Revision: https://reviews.llvm.org/D126386
2022-04-29Reapply [CodeGen][ARM] Enable Swing Module Scheduling for ARMDavid Penry1-7/+10
Fixed "private field is not used" warning when compiled with clang. original commit: 28d09bbbc3d09c912b54a4d5edb32cab7de32a6f reverted in: fa49021c68ef7a7adcdf7b8a44b9006506523191 ------ This patch permits Swing Modulo Scheduling for ARM targets turns it on by default for the Cortex-M7. The t2Bcc instruction is recognized as a loop-ending branch. MachinePipeliner is extended by adding support for "unpipelineable" instructions. These instructions are those which contribute to the loop exit test; in the SMS papers they are removed before creating the dependence graph and then inserted into the final schedule of the kernel and prologues. Support for these instructions was not previously necessary because current targets supporting SMS have only supported it for hardware loop branches, which have no loop-exit-contributing instructions in the loop body. The current structure of the MachinePipeliner makes it difficult to remove/exclude these instructions from the dependence graph. Therefore, this patch leaves them in the graph, but adds a "normalization" method which moves them in the schedule to stage 0, which causes them to appear properly in kernel and prologues. It was also necessary to be more careful about boundary nodes when iterating across successors in the dependence graph because the loop exit branch is now a non-artificial successor to instructions in the graph. In additional, schedules with physical use/def pairs in the same cycle should be treated as creating an invalid schedule because the scheduling logic doesn't respect physical register dependence once scheduled to the same cycle. Reviewed By: dmgreen Differential Revision: https://reviews.llvm.org/D122672
2022-04-28Revert "[CodeGen][ARM] Enable Swing Module Scheduling for ARM"David Penry1-10/+7
This reverts commit 28d09bbbc3d09c912b54a4d5edb32cab7de32a6f while I investigate a buildbot failure.
2022-04-28[CodeGen][ARM] Enable Swing Module Scheduling for ARMDavid Penry1-7/+10
This patch permits Swing Modulo Scheduling for ARM targets turns it on by default for the Cortex-M7. The t2Bcc instruction is recognized as a loop-ending branch. MachinePipeliner is extended by adding support for "unpipelineable" instructions. These instructions are those which contribute to the loop exit test; in the SMS papers they are removed before creating the dependence graph and then inserted into the final schedule of the kernel and prologues. Support for these instructions was not previously necessary because current targets supporting SMS have only supported it for hardware loop branches, which have no loop-exit-contributing instructions in the loop body. The current structure of the MachinePipeliner makes it difficult to remove/exclude these instructions from the dependence graph. Therefore, this patch leaves them in the graph, but adds a "normalization" method which moves them in the schedule to stage 0, which causes them to appear properly in kernel and prologues. It was also necessary to be more careful about boundary nodes when iterating across successors in the dependence graph because the loop exit branch is now a non-artificial successor to instructions in the graph. In additional, schedules with physical use/def pairs in the same cycle should be treated as creating an invalid schedule because the scheduling logic doesn't respect physical register dependence once scheduled to the same cycle. Reviewed By: dmgreen Differential Revision: https://reviews.llvm.org/D122672
2022-03-16[NFC][CodeGen] Rename some functions in MachineInstr.h and remove duplicated ↵Shengchen Kan1-6/+6
comments
2022-03-16Cleanup codegen includesserge-sans-paille1-0/+1
This is a (fixed) recommit of https://reviews.llvm.org/D121169 after: 1061034926 before: 1063332844 Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup Differential Revision: https://reviews.llvm.org/D121681
2022-03-10Revert "Cleanup codegen includes"Nico Weber1-1/+0
This reverts commit 7f230feeeac8a67b335f52bd2e900a05c6098f20. Breaks CodeGenCUDA/link-device-bitcode.cu in check-clang, and many LLVM tests, see comments on https://reviews.llvm.org/D121169
2022-03-10Cleanup codegen includesserge-sans-paille1-0/+1
after: 1061034926 before: 1063332844 Differential Revision: https://reviews.llvm.org/D121169
2022-01-07[llvm] Use true/false instead of 1/0 (NFC)Kazu Hirata1-2/+2
Identified with modernize-use-bool-literals.
2021-11-28[llvm] Use range-based for loops (NFC)Kazu Hirata1-2/+1
2021-11-26[Target] Use range-based for loops (NFC)Kazu Hirata1-2/+1
2021-11-11[CodeGen, Target] Use MachineRegisterInfo::use_operands (NFC)Kazu Hirata1-12/+5
2021-11-11[CodeGen] Use MachineInstr::operands (NFC)Kazu Hirata1-6/+4
2021-11-10[llvm] Use make_early_inc_range (NFC)Kazu Hirata1-11/+4
2021-11-06[llvm] Use llvm::reverse (NFC)Kazu Hirata1-5/+3
2021-11-01[CodeGen] Use make_early_inc_range (NFC)Kazu Hirata1-12/+12
2021-10-31[CodeGen, Target] Use MachineBasicBlock::terminators (NFC)Kazu Hirata1-5/+3
2021-10-26[CodeGen, Hexagon] Use MachineBasicBlock::phis (NFC)Kazu Hirata1-12/+7
2021-06-25[ModuloSchedule] Pass loop block explicitly to kernel rewriter.Hendrik Greving1-5/+5
This change is NFC upstream. We pass in the loop's block to the kernel rewriter explicitly, instead of assuming it's the loop's top block. This change is made for downstream targets where this assumption doesn't hold. Differential Revision: https://reviews.llvm.org/D104811
2020-09-17ModuloSchedule.cpp - remove unnecessary includes. NFCI.Simon Pilgrim1-2/+0
Already included in ModuloSchedule.h
2020-06-30[ModuloSchedule] Make PeelingModuloScheduleExpander inheritable.Hendrik Greving1-6/+6
Basically a NFC, but allows subclasses access to the entire PeelingModuloScheduleExpander class. We are doing this to allow backends, particularly one that are not necessarily upstreamed, to inherit from PeelingModuloScheduleExpander and access its basic structures. Renames Info into LoopInfo for consistency in PeelingModuloScheduleExpander. Differential Revision: https://reviews.llvm.org/D82673
2020-06-25LiveIntervals.h.h - reduce AliasAnalysis.h include to forward declaration. NFC.Simon Pilgrim1-0/+1
Fix implicit include dependencies in source files and replace legacy AliasAnalysis typedef with AAResults where necessary.
2020-06-08[ModuloSchedule] Support instructions with > 1 destination when walking ↵Hendrik Greving1-3/+4
canonical use. Fixes a minor bug that led to finding the wrong register if the definition had more than one register destination.
2020-05-29[ModuloSchedule] Allow illegal phis to be moved across stages.Hendrik Greving1-12/+15
Fixes a trivial but impactful bug where we did not move illegal phis across stages. This led to incorrect mappings in certain cases.
2020-05-21[ModuloSchedule] Add missing comma.Hendrik Greving1-1/+1
This is a test commit as per Chris to verify commit access. Thanks!
2020-05-21[ModuloSchedule] Trivial fix for instruction with more than one destination ↵Thomas Raoux1-2/+2
in modulo peeler. When moving an instruction into a block where it was referenced by a phi when peeling, refer to the phi's register number and assert that the instruction has it in its destinations. This way, it also covers instructions with more than one destination. Patch by Hendrik Greving! Differential Revision: https://reviews.llvm.org/D80027
generated by cgit v1.1 at 2025-09-30 17:32:33 +0000