aboutsummaryrefslogtreecommitdiff
path: root/llvm/lib/CodeGen/MachineVerifier.cpp
AgeCommit message (Collapse)AuthorFilesLines
2021-08-19[AArch64][GlobalISel] Add G_VECREDUCE fewerElements support for full ↵Amara Emerson1-3/+0
scalarization. For some reductions like G_VECREDUCE_OR on AArch64, we need to scalarize completely if the source is <= 64b. This change adds support for that in the legalizer. If the source has a pow-2 num elements, then we can do a tree reduction using the scalar operation in the individual elements. Otherwise, we just create a sequential chain of operations. For AArch64, we only need to scalarize if the input is <64b. If it's great than 64b then we can first do a fewElements step to 64b, taking advantage of vector instructions until we reach the point of scalarization. I also had to relax the verifier checks for reductions because the intrinsics support <1 x EltTy> types, which we lower to scalars for GlobalISel. Differential Revision: https://reviews.llvm.org/D108276
2021-08-18[GlobalISel] Add G_ISNANJessica Paquette1-0/+19
Add a generic opcode equivalent to the `llvm.isnan` intrinsic + MachineVerifier support for it. We need an opcode here because we may want target-specific lowering later on. Differential Revision: https://reviews.llvm.org/D108222
2021-08-13[NFC] Rename AttributeList::hasFnAttribute() -> hasFnAttr()Arthur Eubanks1-1/+1
This is more consistent with similar methods.
2021-07-21[MachineVerifier] Make INSERT_SUBREG diagnostic respect operand 2 subregsJon Roelofs1-2/+5
This came out of post-commit review: https://reviews.llvm.org/D105953#inline-1012919 Thanks uabelho!
2021-07-20[MachineVerifier] Diagnose invalid INSERT_SUBREGsJon Roelofs1-0/+10
Differential revision: https://reviews.llvm.org/D105953
2021-07-16Revert "[MachineVerifier] Diagnose invalid INSERT_SUBREGs"Jon Roelofs1-9/+0
This reverts commit dd57ba1a17b93dbe211d04cb2d4de5f6dc898d60. It broke some tests: http://45.33.8.238/linux/51314/step_12.txt
2021-07-16[MachineVerifier] Diagnose invalid INSERT_SUBREGsJon Roelofs1-0/+9
Differential revision: https://reviews.llvm.org/D105953
2021-06-30[GISel] Support llvm.memcpy.inlineJon Roelofs1-0/+9
Differential revision: https://reviews.llvm.org/D105072
2021-04-28GlobalISel: Relax verification of physical register copy typesMatt Arsenault1-15/+41
This was picking a concrete size for a physical register, and enforcing exact match on the virtual register's type size. Some targets add multiple types to a register class, and some are smaller than the full bit width. For example x86 adds f32 to 128-bit xmm registers, and AMDGPU adds i16/f16 to 32-bit registers. It might be better to represent these cases as a copy of the full register and an extraction of the subpart, but a lot of code assumes you can directly copy. This will help fix the current usage of the DAG calling convention infrastructure which is incompatible with how GlobalISel is now using it. The API is somewhat cumbersome here, but I just mirrored the existing functions, except now with LLTs (and allow returning null on failure, unlike the MVT version). I think the concept of selecting register classes based on type is flawed to begin with, but I'm trying to keep this compatible with the existing handling.
2021-04-21[CSSPGO] Exclude pseudo probe from slotindex verification.Hongtao Yu1-1/+1
2021-04-20MachineVerifier: Continue reporting errors for copiesMatt Arsenault1-2/+0
This was skipping verification of later copies, but generally the verifier tries to report as many things wrong as possible in the function.
2021-04-19[Greedy RA] Add a check to MachineVerifierSerguei Katkov1-0/+9
If Virtual Register is alive in landing pad its def must be before the call causing the exception or it should be statepoint instruction itself and in this case def actually means the relocation of gc pointer and is alive in landing pad. The test shows the triggering this check for an option under development use-registers-for-gc-values-in-landing-pad which is off by default until it is functionally correct. Reviewers: reames, void, jyknight, nickdesaulniers, efriedma, arsenm, rnk Reviewed By: rnk Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D100525
2021-03-25[GlobalISel] Add G_ROTR and G_ROTL opcodes for rotates.Amara Emerson1-0/+11
Differential Revision: https://reviews.llvm.org/D99383
2021-03-25[AArch64][GlobalISel] Emit bzero on DarwinJessica Paquette1-4/+6
Darwin platforms for both AArch64 and X86 can provide optimized `bzero()` routines. In this case, it may be preferable to use `bzero` in place of a memset of 0. This adds a G_BZERO generic opcode, similar to G_MEMSET et al. This opcode can be generated by platforms which may want to use bzero. To emit the G_BZERO, this adds a pre-legalize combine for AArch64. The conditions for this are largely a port of the bzero case in `AArch64SelectionDAGInfo::EmitTargetCodeForMemset`. The only difference in comparison to the SelectionDAG code is that, when compiling for minsize, this will fire for all memsets of 0. The original code notes that it's not beneficial to do this for small memsets; however, using bzero here will save a mov from wzr. For minsize, I think that it's preferable to prioritise omitting the mov. This also fixes a bug in the libcall legalization code which would delete instructions which could not be legalized. It also adds a check to make sure that we actually get a libcall name. Code size improvements (Darwin): - CTMark -Os: -0.0% geomean (-0.1% on pairlocalalign) - CTMark -Oz: -0.2% geomean (-0.5% on bullet) Differential Revision: https://reviews.llvm.org/D99358
2021-03-19[GlobalISel] Add G_SBFX + G_UBFX (bitfield extraction opcodes)Jessica Paquette1-0/+11
There is a bunch of similar bitfield extraction code throughout *ISelDAGToDAG. E.g, ARMISelDAGToDAG, AArch64ISelDAGToDAG, and AMDGPUISelDAGToDAG all contain code that matches a bitfield extract from an and + right shift. Rather than duplicating code in the same way, this adds two opcodes: - G_UBFX (unsigned bitfield extract) - G_SBFX (signed bitfield extract) They work like this ``` %x = G_UBFX %y, %lsb, %width ``` Where `lsb` and `width` are - The least-significant bit of the extraction - The width of the extraction This will extract `width` bits from `%y`, starting at `lsb`. G_UBFX zero-extends the result, while G_SBFX sign-extends the result. This should allow us to use the combiner to match the bitfield extraction patterns rather than duplicating pattern-matching code in each target. Differential Revision: https://reviews.llvm.org/D98464
2021-03-08[M68k][MIR](2/8) Changes in the target-independent MIR partMin-Yih Hsu1-3/+6
- Add new callback in `TargetInstrInfo` -- `isPCRelRegisterOperandLegal` -- to query whether pc-rel register MachineOperand is legal. - Add new function to search DebugLoc in a reverse ordering Authors: myhsu, m4yers, glaubitz Differential Revision: https://reviews.llvm.org/D88386
2021-03-01GlobalISel: Verify G_CONCAT_VECTORS has at least 2 sourcesMatt Arsenault1-0/+4
2021-02-17[GlobalISel] Add G_ASSERT_SEXTJessica Paquette1-8/+19
This adds a G_ASSERT_SEXT opcode, similar to G_ASSERT_ZEXT. This instruction signifies that an operation was already sign extended from a smaller type. This is useful for functions with sign-extended parameters. E.g. ``` define void @foo(i16 signext %x) { ... } ``` This adds verifier, regbankselect, and instruction selection support for G_ASSERT_SEXT equivalent to G_ASSERT_ZEXT. Differential Revision: https://reviews.llvm.org/D96890
2021-02-14[llvm] Use llvm::is_contained (NFC)Kazu Hirata1-6/+2
2021-02-12[GlobalISel] Simpler verification of G_SEXT_INREG and G_ASSERT_ZEXTJay Foad1-5/+0
There's no need to call verifyVectorElementMatch since we already know that the source and destination types are identical. Differential Revision: https://reviews.llvm.org/D96589
2021-01-28[GlobalISel] Add G_ASSERT_ZEXTJessica Paquette1-1/+37
This adds a generic opcode which communicates that a type has already been zero-extended from a narrower type. This is intended to be similar to AssertZext in SelectionDAG. For example, ``` %x_was_extended:_(s64) = G_ASSERT_ZEXT %x, 16 ``` Signifies that the top 48 bits of %x are known to be 0. This is useful in cases like this: ``` define i1 @zeroext_param(i8 zeroext %x) { %cmp = icmp ult i8 %x, -20 ret i1 %cmp } ``` In AArch64, `%x` must use a 32-bit register, which is then truncated to a 8-bit value. If we know that `%x` is already zero-ed out in the relevant high bits, we can avoid the truncate. Currently, in GISel, this looks like this: ``` _zeroext_param: and w8, w0, #0xff ; We don't actually need this! cmp w8, #236 cset w0, lo ret ``` While SDAG does not produce the truncation, since it knows that it's unnecessary: ``` _zeroext_param: cmp w0, #236 cset w0, lo ret ``` This patch - Adds G_ASSERT_ZEXT - Adds MIRBuilder support for it - Adds MachineVerifier support for it - Documents it It also puts G_ASSERT_ZEXT into its own class of "hint instruction." (There should be a G_ASSERT_SEXT in the future, maybe a G_ASSERT_ALIGN as well.) This allows us to skip over hints in the legalizer etc. These can then later be selected like COPY instructions or removed. Differential Revision: https://reviews.llvm.org/D95564
2021-01-21[CodeGen] Use llvm::append_range (NFC)Kazu Hirata1-2/+1
2021-01-19[llvm] Use llvm::all_of (NFC)Kazu Hirata1-10/+9
2021-01-13[Verifier] Add tied-ness verification to statepoint intsructionSerguei Katkov1-0/+16
Reviewers: reames, dantrushin Reviewed By: reames, dantrushin Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D94483
2021-01-13[Verifier] Extend statepoint verifier to cover more constantsSerguei Katkov1-0/+7
Also old mir tests are updated to meet last changes in STATEPOINT format. Reviewers: reames, dantrushin Reviewed By: reames, dantrushin Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D94482
2020-12-10[ARM][RegAlloc] Add t2LoopEndDecDavid Green1-0/+10
We currently have problems with the way that low overhead loops are specified, with LR being spilled between the t2LoopDec and the t2LoopEnd forcing the entire loop to be reverted late in the backend. As they will eventually become a single instruction, this patch introduces a t2LoopEndDec which is the combination of the two, combined before registry allocation to make sure this does not fail. Unfortunately this instruction is a terminator that produces a value (and also branches - it only produces the value around the branching edge). So this needs some adjustment to phi elimination and the register allocator to make sure that we do not spill this LR def around the loop (needing to put a spill after the terminator). We treat the loop very carefully, making sure that there is nothing else like calls that would break it's ability to use LR. For that, this adds a isUnspillableTerminator to opt in the new behaviour. There is a chance that this could cause problems, and so I have added an escape option incase. But I have not seen any problems in the testing that I've tried, and not reverting Low overhead loops is important for our performance. If this does work then we can hopefully do the same for t2WhileLoopStart and t2DoLoopStart instructions. This patch also contains the code needed to convert or revert the t2LoopEndDec in the backend (which just needs a subs; bne) and the code pre-ra to create them. Differential Revision: https://reviews.llvm.org/D91358
2020-10-20[NFC][MC] Use [MC]Register in MachineVerifierMircea Trofin1-55/+62
Differential Revision: https://reviews.llvm.org/D89815
2020-10-15[GlobalISel] Remove scalar src from non-sequential fadd/fmul reductions.Amara Emerson1-5/+5
It's probably better to split these into separate G_FADD/G_FMUL + G_VECREDUCE operations in the translator rather than carrying the scalar around. The majority of the time it'll get simplified away as the scalars are probably identity values. Differential Revision: https://reviews.llvm.org/D89150
2020-10-14[DebugInstrRef] Parse debug instruction-references from/to MIRJeremy Morse1-0/+20
This patch defines the MIR format for debug instruction references: it's an integer trailing an instruction, marked out by "debug-instr-number", much like how "debug-location" identifies the DebugLoc metadata of an instruction. The instruction number is stored directly in a MachineInstr. Actually referring to an instruction comes in a later patch, but is done using one of these instruction numbers. I've added a round-trip test and two verifier checks: that we don't label meta-instructions as generating values, and that there are no duplicates. Differential Revision: https://reviews.llvm.org/D85746
2020-10-08[GlobalISel] Add G_VECREDUCE_* opcodes for vector reductions.Amara Emerson1-0/+34
These mirror the IR and SelectionDAG intrinsics & nodes. Opcodes added: G_VECREDUCE_SEQ_FADD G_VECREDUCE_SEQ_FMUL G_VECREDUCE_FADD G_VECREDUCE_FMUL G_VECREDUCE_FMAX G_VECREDUCE_FMIN G_VECREDUCE_ADD G_VECREDUCE_MUL G_VECREDUCE_AND G_VECREDUCE_OR G_VECREDUCE_XOR G_VECREDUCE_SMAX G_VECREDUCE_SMIN G_VECREDUCE_UMAX G_VECREDUCE_UMIN Differential Revision: https://reviews.llvm.org/D88750
2020-09-19Fix some clang-tidy bugprone-argument-comment issuesFangrui Song1-1/+1
2020-09-16[NFC][Regalloc] accessors for 'reg' and 'weight'Mircea Trofin1-4/+4
Also renamed the fields to follow style guidelines. Accessors help with readability - weight mutation, in particular, is easier to follow this way. Differential Revision: https://reviews.llvm.org/D87725
2020-08-27[ARM] Make MachineVerifier more strict about terminatorsSam Parker1-3/+1
Fix the ARM backend's analyzeBranch so it doesn't ignore predicated return instructions, and make the MachineVerifier rule more strict. Differential Revision: https://reviews.llvm.org/D40061
2020-08-26GlobalISel: Add generic instructions for memory intrinsicsMatt Arsenault1-14/+56
AArch64, X86 and Mips currently directly consumes these and custom lowering to produce a libcall, but really these should follow the normal legalization process through the libcall/lower action.
2020-08-13[NewPM][CodeGen] Add machine code verification callbackYuanfang Chen1-0/+13
D83608 need this. Reviewed By: aeubanks Differential Revision: https://reviews.llvm.org/D85916
2020-08-11NFC. Constify MachineVerifier::verify parameterYuanfang Chen1-2/+2
2020-08-01[MachineVerifier] Refactor calcRegsPassed. NFCEvgeny Leviant1-52/+17
Patch improves performance of verify-machineinstrs pass up to 10x. Differential revision: https://reviews.llvm.org/D84105
2020-07-29[MachineVerifier] Handle the PHI node for verifyLiveVariables()Kang Zhang1-1/+19
Summary: When doing MachineVerifier for LiveVariables, the MachineVerifier pass will calculate the LiveVariables, and compares the result with the result livevars pass gave. If they are different, verifyLiveVariables() will give error. But when we calculate the LiveVariables in MachineVerifier, we don't consider the PHI node, while livevars considers. This patch is to fix above bug. Reviewed By: bjope Differential Revision: https://reviews.llvm.org/D80274
2020-07-16[Statepoint] Fix bug found by sanitaizer.Denis Antrushin1-3/+0
Statepoint has no static operands, so it cannot be verified against MCInstrDescr. Revert NumDefs change introduced by ef658ebd629.
2020-07-17MIR Statepoint refactoring. Part 1: Basic MI level changes.Denis Antrushin1-0/+3
Basic support for variadic-def MIR Statepoint: - Change TableGen STATEPOINT description to variadic out list (For self-documentation purpose; by itself it does not affect code generation in any way). - Update StatepointOpers helper class to handle variadic defs. - Update MachineVerifier to properly handle them, too. With this change, new Statepoint instruction can be passed through backend (excluding ISEL) without errors. Full change set is available at D81603. Reviewed By: reames Differential Revision: https://reviews.llvm.org/D81645
2020-07-08GlobalISel: Verify G_BITCAST changes the typeMatt Arsenault1-0/+4
Updated the AArch64 tests the best I could with my vague, inferred understanding of AArch64 register banks. As far as I can tell, there is only one 32-bit/64-bit type which will use the gpr register bank, so we have to use the fpr bank for the other operand.
2020-07-01Change the INLINEASM_BR MachineInstr to be a non-terminating instruction.James Y Knight1-2/+1
Before this instruction supported output values, it fit fairly naturally as a terminator. However, being a terminator while also supporting outputs causes some trouble, as the physreg->vreg COPY operations cannot be in the same block. Modeling it as a non-terminator allows it to be handled the same way as invoke is handled already. Most of the changes here were created by auditing all the existing users of MachineBasicBlock::isEHPad() and MachineBasicBlock::hasEHPadSuccessor(), and adding calls to isInlineAsmBrIndirectTarget or mayHaveInlineAsmBr, as appropriate. Reviewed By: nickdesaulniers, void Differential Revision: https://reviews.llvm.org/D79794
2020-06-30GlobalISel: Disallow undef generic virtual register usesMatt Arsenault1-0/+9
With an undef operand, it's possible for getVRegDef to fail and return null. This is an edge case very little code bothered to consider. Proper gMIR should use G_IMPLICIT_DEF instead. I initially tried to apply this restriction to all SSA MIR, so then getVRegDef would never fail anywhere. However, ProcessImplicitDefs does technically run while the function is in SSA. ProcessImplicitDefs and DetectDeadLanes would need to either move, or a new pseudo-SSA type of function property would need to be introduced.
2020-06-15[NFC] Add braces to if-statement in MachineVerifierDominik Montada1-1/+2
2020-06-15[MachineVerifier][GlobalISel] Check that branches have a MBB operand or are ↵Dominik Montada1-0/+16
declared indirect. Add missing properties to G_BRJT, G_BRINDIRECT Summary: Teach MachineVerifier to check branches for MBB operands if they are not declared indirect. Add `isBarrier`, `isIndirectBranch` to `G_BRINDIRECT` and `G_BRJT`. Without these, `MachineInstr.isConditionalBranch()` was giving a false-positive for those instructions. Reviewers: aemerson, qcolombet, dsanders, arsenm Reviewed By: dsanders Subscribers: hiraditya, wdng, simoncook, s.egerton, arsenm, rovka, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D81587
2020-06-09GlobalISel: Remove redundant check in verifierMatt Arsenault1-3/+0
This was already checked earlier for all instructions.
2020-06-09[MachineVerifier] Add TiedOpsRewritten flag to fix verify two-address errorKang Zhang1-3/+10
Summary: Currently, MachineVerifier will attempt to verify that tied operands satisfy register constraints as soon as the function is no longer in SSA form. However, PHIElimination will take the function out of SSA form while TwoAddressInstructionPass will actually rewrite tied operands to match the constraints. PHIElimination runs first in the pipeline. Therefore, whenever the MachineVerifier is run after PHIElimination, it will encounter verification errors on any tied operands. This patch adds a function property called TiedOpsRewritten that will be set by TwoAddressInstructionPass and will control when the verifier checks tied operands. Reviewed By: nemanjai Differential Revision: https://reviews.llvm.org/D80538
2020-06-06Simplify MachineVerifier's block-successor verification.James Y Knight1-91/+47
There's two properties we want to verify: 1. That the successors returned by analyzeBranch are in the CFG successor list, and 2. That there are no extraneous successors are in the CFG successor list. The previous implementation mostly accomplished this, but in a very convoluted manner. Differential Revision: https://reviews.llvm.org/D79793
2020-05-28[MachineVerifier] Verify that a DBG_VALUE has a debug locationVedant Kumar1-0/+7
Summary: Verify that each DBG_VALUE has a debug location. This is required by LiveDebugValues, and perhaps by other late passes. There's an exception for tests: lots of tests use a two-operand form of DBG_VALUE for convenience. There's no reason to prevent that. This is an extension of D80665, but there's no dependency. Reviewers: aprantl, jmorse, davide, chrisjackson Subscribers: hiraditya, asb, rbar, johnrusso, simoncook, sabuasal, niosHD, jrtc27, MaskRay, zzheng, edward-jones, rogfer01, MartinMosbeck, brucehoult, the_o, PkmX, jocewei, Jim, lenary, s.egerton, pzheng, sameer.abuasal, apazos, luismarques, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D80670
2020-05-26GlobalISel: Merge G_PTR_MASK with llvm.ptrmask intrinsicMatt Arsenault1-0/+16
Confusingly, these were unrelated and had different semantics. The G_PTR_MASK instruction predates the llvm.ptrmask intrinsic, but has a different format. G_PTR_MASK only allows clearing the low bits of a pointer, and only a constant number of bits. The ptrmask intrinsic allows an arbitrary mask. Replace G_PTR_MASK to match the intrinsic. Only selects the cases that look like the old instruction. More work is needed to select the general case. Also new legalization code is still needed to deal with the case where the incoming mask size does not match the pointer size, which has a specified behavior in the langref.
generated by cgit v1.1 at 2025-10-07 10:46:15 +0000