aboutsummaryrefslogtreecommitdiff
path: root/llvm/lib/CodeGen/CodeGenPrepare.cpp
AgeCommit message (Collapse)AuthorFilesLines
2018-03-29[IR][CodeGen] Remove dependency on EVT from IR/Function.cpp. Move EVT to ↵Craig Topper1-1/+1
CodeGen layer. Currently EVT is in the IR layer only because of Function.cpp needing a very small piece of the functionality of EVT::getEVTString(). The rest of EVT is used in codegen making CodeGen a better place for it. The previous code converted a Type* to EVT and then called getEVTString. This was only expected to handle the primitive types from Type*. Since there only a few primitive types, we can just print them as strings directly. Differential Revision: https://reviews.llvm.org/D45017 llvm-svn: 328806
2018-03-28Plumb useAA through TargetTransformInfo to remove Transforms->CodeGen header ↵David Blaikie1-2/+1
dependency Thanks to echristo for the pointers on direction. llvm-svn: 328737
2018-03-23Fix layering by moving ValueTypes.h from CodeGen to IRDavid Blaikie1-1/+1
ValueTypes.h is implemented in IR already. llvm-svn: 328397
2018-03-23Fix layering of MachineValueType.h by moving it from CodeGen to SupportDavid Blaikie1-1/+1
This is used by llvm tblgen as well as by LLVM Targets, so the only common place is Support for now. (maybe we need another target for these sorts of things - but for now I'm at least making them correct & we can make them better if/when people have strong feelings) llvm-svn: 328395
2018-03-21Fix a couple of layering violations in TransformsDavid Blaikie1-1/+1
Remove #include of Transforms/Scalar.h from Transform/Utils to fix layering. Transforms depends on Transforms/Utils, not the other way around. So remove the header and the "createStripGCRelocatesPass" function declaration (& definition) that is unused and motivated this dependency. Move Transforms/Utils/Local.h into Analysis because it's used by Analysis/MemoryBuiltins.cpp. llvm-svn: 328165
2018-03-20[CGP] Avoid segmentation fault when doing PHI node simplificationsBjorn Pettersson1-61/+71
Summary: Made PHI node simplifiations more robust in several ways: - Minor refactoring to let the SimplificationTracker own the sets with new PHI/Select nodes that are introduced. This is maybe not mapping to the original intention with the SimplificationTracker, but IMHO it encapsulates the logic behind those sets a little bit better. - MatchPhiNode can sometimes populate the Matched set with several entries, where it maps one PHI node to different candidates for replacement. The Matched set is changed into a SmallSetVector to make sure we get a deterministic iteration when doing the replacements. - As described above we may get several different replacements for a single PHI node. The loop in MatchPhiSet that is doing the replacements could end up calling eraseFromParent several times for the same PHI node, resulting in segmentation faults. This problem was supposed to be fixed in rL327250, but due to the non-determinism(?) it only appeared to be fixed (I still got crashes sometime when turning on/off -print-after-all etc to get different iteration order in the DenseSets). With this patch we follow the deterministic ordering in the Matched set when replacing the PHI nodes. If we find a new replacement for an already replaced PHI node we replace the new replacement by the old replacement instead. This is quite similar to what happened in the rl327250 patch, but here we also recursively verify that the old replacement hasn't been replaced already. - It was really hard to track down the fault described above (segementation fault due to doing eraseFromParent multiple times for the same instruction). The fault was intermittent and small changes in the code, or simply turning on -print-after-all etc could make the problem go away. This was basically due to the iteration over PhiNodesToMatch in MatchPhiSet no being deterministic. Therefore I've changed the data structure for the SimplificationTracker::AllPhiNodes into an SmallSetVector. This gives a deterministic behavior. Reviewers: skatkov, john.brawn Reviewed By: skatkov Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D44571 llvm-svn: 327961
2018-03-13[CodeGenPrepare] Respect endianness in splitMergedValStore.Jonas Paulsson1-1/+2
splitMergedValStore will split a store into two if target prefers this, or if -force-split-store is passed. This patch adds the missing handling for endianness in this function along with a test case. Review: Eli Friedman https://reviews.llvm.org/D44396 llvm-svn: 327375
2018-03-12[CGP] Fix the remove of matched phis in complex addressing modeSerguei Katkov1-1/+13
When we replace the Phi we created with matched ones it is possible that there are two identical phi nodes in IR. And matcher is smart enough to find that new created phi matches both of them. So we try to replace our phi node with matched ones twice and what is bad we delete our phi node twice causing a crash. As soon as we found that we have two identical Phi nodes it makes sense to do a clean-up and replace one phi node by other one. The patch implements it. Reviewers: john.brawn, reames Reviewed By: john.brawn Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D43758 llvm-svn: 327250
2018-02-14Adding a width of the GEP index to the Data Layout.Elena Demikhovsky1-1/+1
Making a width of GEP Index, which is used for address calculation, to be one of the pointer properties in the Data Layout. p[address space]:size:memory_size:alignment:pref_alignment:index_size_in_bits. The index size parameter is optional, if not specified, it is equal to the pointer size. Till now, the InstCombiner normalized GEPs and extended the Index operand to the pointer width. It works fine if you can convert pointer to integer for address calculation and all registered targets do this. But some ISAs have very restricted instruction set for the pointer calculation. During discussions were desided to retrieve information for GEP index from the Data Layout. http://lists.llvm.org/pipermail/llvm-dev/2018-January/120416.html I added an interface to the Data Layout and I changed the InstCombiner and some other passes to take the Index width into account. This change does not affect any in-tree target. I added tests to cover data layouts with explicitly specified index size. Differential Revision: https://reviews.llvm.org/D42123 llvm-svn: 325102
2018-01-31[CodeGenPrepare] Improve source and dest alignments of memory intrinsics ↵Daniel Neilson1-5/+8
independently Summary: This change is part of step five in the series of changes to remove alignment argument from memcpy/memmove/memset in favour of alignment attributes. In particular, this changes the CodeGenPrepare pass to be more aggressive in improving the source and destination alignments of memcpy/memmove/memset by exploiting our new ability to record independent alignments for each argument. Steps: Step 1) Remove alignment parameter and create alignment parameter attributes for memcpy/memmove/memset. ( rL322965, rC322964, rL322963 ) Step 2) Expand the IRBuilder API to allow creation of memcpy/memmove with differing source and dest alignments. ( rL323597 ) Step 3) Update Clang to use the new IRBuilder API. ( rC323617 ) Step 4) Update Polly to use the new IRBuilder API. ( rL323618 ) Step 5) Update LLVM passes that create memcpy/memmove calls to use the new IRBuilder API, and those that use use MemIntrinsicInst::[get|set]Alignment() to use [get|set]DestAlignment() and [get|set]SourceAlignment() instead. ( rL323886 ) Step 6) Remove the single-alignment IRBuilder API for memcpy/memmove, and the MemIntrinsicInst::[get|set]Alignment() methods. Reference http://lists.llvm.org/pipermail/llvm-dev/2015-August/089384.html http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20151109/312083.html llvm-svn: 323891
2018-01-26[CGP] Re-enable Select in complex addressing mode.Serguei Katkov1-1/+1
Switch Select handling on after fixing two bugs: rL323192 and rL323497. llvm-svn: 323498
2018-01-23[CGP] Fix the GV handling in complex addressing modeSerguei Katkov1-15/+21
If in complex addressing mode the difference is in GV then base reg should not be installed because we plan to use base reg as a merge point of different GVs. This is a fix for PR35980. Reviewers: reames, john.brawn, santosh Reviewed By: john.brawn Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D42230 llvm-svn: 323192
2018-01-19Revert [CGP] Re-enable Select in complex addressing modeSerguei Katkov1-1/+1
One of buildbots failed. Revert for now till fix the issue. llvm-svn: 322923
2018-01-12[NFC] Change MemIntrinsicInst::setAlignment() to take an unsigned instead of ↵Daniel Neilson1-1/+1
a Constant Summary: In preparation for https://reviews.llvm.org/D41675 this NFC changes this prototype of MemIntrinsicInst::setAlignment() to accept an unsigned instead of a Constant. llvm-svn: 322403
2018-01-12[CGP] Re-enable Select in complex addressing modeSerguei Katkov1-1/+1
Re-enable Select after a couple of fixes. Differential Revision: https://reviews.llvm.org/D40634 llvm-svn: 322358
2018-01-09Tidy some grammar in some commentsEric Christopher1-3/+3
llvm-svn: 322133
2018-01-09[CGP] Fix Complex addressing mode for offsetSerguei Katkov1-1/+6
If the offset is differ in two addressing mode we can continue only if ScaleReg is not set due to we will use it as merge of different offsets. It should fix PR35799 and PR35805. Reviewers: john.brawn, reames Reviewed By: reames Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41227 llvm-svn: 322056
2017-12-30Use phi ranges to simplify code. No functionality change intended.Benjamin Kramer1-37/+21
llvm-svn: 321585
2017-12-20[PGO] Function section hotness prefix should look at all blocksTeresa Johnson1-11/+4
Summary: The function section prefix for PGO based layout (e.g. hot/unlikely) should look at the hotness of all blocks not just the entry BB. A function with a cold entry but a very hot loop should be placed in the hot section, for example, so that it is located close to other hot functions it may call. For SamplePGO it was already looking at the branch weights on calls, and I made that code conditional on whether this is SamplePGO since it was essentially a noop for instrumentation PGO anyway. Reviewers: davidxl Subscribers: eraman, llvm-commits Differential Revision: https://reviews.llvm.org/D41395 llvm-svn: 321197
2017-12-19[CGP] Format. NFCHaicheng Wu1-1/+1
Clang-format. llvm-svn: 321107
2017-12-18[CGP] Fix the handling select inst in complex addressing modeSerguei Katkov1-2/+2
When we put the value in select placeholder we must pass the value through simplification tracker due to the value might be already simplified and erased. This is a fix for PR35658. Reviewers: john.brawn, uabelho Reviewed By: john.brawn Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41251 llvm-svn: 320956
2017-12-13Revert "[CGP] Enable select in complex addr mode"Serguei Katkov1-1/+1
Causes: Assertion `ScaledReg == nullptr' failed. This actually a revert of rL320551. llvm-svn: 320553
2017-12-13[CGP] Enable select in complex addr modeSerguei Katkov1-1/+1
Enable select instruction handling in complex addr modes. Reviewers: john.brawn, reames, aaboud Reviewed By: reames Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D40634 llvm-svn: 320551
2017-12-04Move splitIndirectCriticalEdges() to BasicBlockUtils.h.Hiroshi Yamauchi1-159/+1
Summary: Move splitIndirectCriticalEdges() from CodeGenPrepare to BasicBlockUtils.h so that it can be called from other places. Reviewers: davidxl Reviewed By: davidxl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D40750 llvm-svn: 319689
2017-11-29[CGP] Enable complex addr modeSerguei Katkov1-1/+1
Enable complex addr modes after two critical fixes: rL319109 and rL319292 llvm-svn: 319302
2017-11-29[CGP] Fix common type handling in optimizeMemoryInstSerguei Katkov1-6/+10
If common type is different we should bail out due to we will not be able to create a select or Phi of these values. Basically it is done in ExtAddrMode::compare however it does not work if we handle the null first and then two values of different types. so add a check in initializeMap as well. The check in ExtAddrMode::compare is used as earlier bail out. Reviewers: reames, john.brawn Reviewed By: john.brawn Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D40479 llvm-svn: 319292
2017-11-27[CGP] Fix handling of null pointer values in optimizeMemoryInstJohn Brawn1-9/+7
The current way that trivial addressing modes are detected incorrectly thinks that null pointers are non-trivial, leading to an infinite loop where we keep duplicating the same select. Fix this by aware of null when deciding if an addressing mode is trivial. Differential Revision: https://reviews.llvm.org/D40447 llvm-svn: 319019
2017-11-24[CodeGenPrepare] Check that erased sunken address are not reusedSimon Dardis1-5/+14
CodeGenPrepare sinks address computations from one basic block to another and attempts to reuse address computations that have already been sunk. If the same address computation appears twice with the first instance as an operand of a load whose result is an operand to a simplifable select, CodeGenPrepare simplifies the select and recursively erases the now dead instructions. CodeGenPrepare then attempts to use the erased address computation for the second load. Fix this by erasing the cached address value if it has zero uses before looking for the address value in the sunken address map. This partially resolves PR35209. Thanks to Alexander Richardson for reporting the issue! This fixed version relands r318032 which was reverted in r318049 due to sanitizer buildbot failures. Reviewers: john.brawn Differential Revision: https://reviews.llvm.org/D39841 llvm-svn: 318956
2017-11-24[CGP] Make optimizeMemoryInst able to combine more kinds of ExtAddrMode fieldsJohn Brawn1-12/+94
This patch extends the recent work in optimizeMemoryInst to make it able to combine more ExtAddrMode fields than just the BaseReg. This fixes some benchmark regressions introduced by r309397, where GVN PRE is hoisting a getelementptr such that it can no longer be combined into the addressing mode of the load or store that uses it. Differential Revision: https://reviews.llvm.org/D38133 llvm-svn: 318949
2017-11-21Revert "[CGP] Enable complex addr mode (2nd attempt)"Serguei Katkov1-1/+1
Revert the patch rl318728 causing buildbot hangs-ups. llvm-svn: 318731
2017-11-21[CGP] Enable complex addr mode (2nd attempt)Serguei Katkov1-1/+1
2nd attempt to enable complex addr modes after fix of the crash by rL318638. llvm-svn: 318728
2017-11-20[CGP] Fix the crash caused by enable of complex addr modeSerguei Katkov1-9/+5
We must collect all AddModes even if they are the same. This is due to Original value is different but we need all original values collected as they are used as anchors in common phi finding. Reviewers: john.brawn, reames Reviewed By: john.brawn Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D40166 llvm-svn: 318638
2017-11-17Fix a bunch more layering of CodeGen headers that are in TargetDavid Blaikie1-2/+2
All these headers already depend on CodeGen headers so moving them into CodeGen fixes the layering (since CodeGen depends on Target, not the other way around). llvm-svn: 318490
2017-11-14[CodeGenPrepare] Disable div bypass when working set size is huge.Easwaran Raman1-3/+4
Summary: Bypass of slow divs based on operand values is currently disabled for -Os. Do the same when profile summary is available and the working set size of the application is huge. This is similar to how loop peeling is guarded by hasHugeWorkingSetSize. In the div bypass case, the generated extra code (and the extra branch) tendss to outweigh the benefits of the bypass. This results in noticeable performance improvement on an internal application. Reviewers: davidxl Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D39992 llvm-svn: 318179
2017-11-13Revert "[CodeGenPrepare] Check that erased sunken address are not reused"Simon Dardis1-6/+1
This reverts commit r318032. The test broke some sanitizer bots. llvm-svn: 318049
2017-11-13[CodeGenPrepare] Check that erased sunken address are not reusedSimon Dardis1-1/+6
CodeGenPrepare sinks address computations from one basic block to another and attempts to reuse address computations that have already been sunk. If the same address computation appears twice with the first instance as an operand of a load whose result is an operand to a simplifable select, CodeGenPrepare simplifies the select and recursively erases the now dead instructions. CodeGenPrepare then attempts to use the erased address computation for the second load. Fix this by erasing the cached address value if it has zero uses before looking for the address value in the sunken address map. This partially resolves PR35209. Thanks to Alexander Richardson for reporting the issue! Reviewers: john.brawn Differential Revision: https://reviews.llvm.org/D39841 llvm-svn: 318032
2017-11-08Revert "[CGP] Enable extending scope of optimizeMemoryInst"Serguei Katkov1-1/+1
Revert the patch r317665 causing buildbot failures. llvm-svn: 317667
2017-11-08[CGP] Enable extending scope of optimizeMemoryInstSerguei Katkov1-1/+1
This patch enables the folding of address computation in memory instruction in case adress is represented by Phi node. The inputs of Phi node might be different in base register. Differential Revision: https://reviews.llvm.org/D36073 llvm-svn: 317665
2017-11-07[CodeGenPrepare] Fix typo in comment. NFCCraig Topper1-1/+1
llvm-svn: 317614
2017-11-07[CGP] Disable Select instruction handling in optimizeMemoryInst. NFCSerguei Katkov1-1/+1
This patch disables the handling of selects in optimization extensing scope of optimizeMemoryInst. The optimization itself is disable by default. The idea here is just to switch optimiztion level step by step. Specifically, first optimization will be enabled only for Phi nodes, then select instructions will be added. In case someone will complain about perfromance it will be easier to detect what part of optimizations is responsible for that. Differential Revision: https://reviews.llvm.org/D36073 llvm-svn: 317555
2017-11-05[CGP] Fix the bug found by asan.Serguei Katkov1-2/+2
Try to fix the asan failure introduced by r317429. llvm-svn: 317431
2017-11-05[CGP] Extends the scope of optimizeMemoryInst optimizationSerguei Katkov1-5/+438
This is an implementation of PR26223. Currently optimizeMemoryInst optimization tries to fold address computation if all possible way to get compute the address are of the form baseGV + base + scale * Index + offset where scale and offset are constants and baseGV, base and Index are exactly the same instructions if defined. The patch extends this optimization to allow different bases. In this case it tries to find/build a Phi node merging all possible bases and use this Phi node as a base for sunk address computation. Also it supports Select instruction on the way. The main motivation for this scope extension is GCRelocateInst. If there is a relocation of derived pointer it will be represented as relocation of base + offset. Also there will be a Phi node merging address computation for relocated derived pointer and derived pointer itself. If we have a Phi node merging original base and relocated base and can fold the address computation of derived pointer then we can potentially reduce the code size and Phi node for derived pointer. The later can have a positive impact to register allocator. Reviewers: efriedma, dberlin, mkazantsev, reames, john.brawn Reviewed By: john.brawn Subscribers: javed.absar, john.brawn, dneilson, llvm-commits Differential Revision: https://reviews.llvm.org/D36073 llvm-svn: 317429
2017-11-03Invoke salvageDebugInfo from CodeGenPrepare's SinkCast()Adrian Prantl1-0/+1
This preserves the debug info for the cast operation in the original location. rdar://problem/33460652 Reapplied r317340 with the test moved into an ARM-specific directory. llvm-svn: 317375
2017-11-03Revert "Invoke salvageDebugInfo from CodeGenPrepare's SinkCast()"Adrian Prantl1-1/+0
This reverts commit 317342 while investigating bot breakage. llvm-svn: 317345
2017-11-03Invoke salvageDebugInfo from CodeGenPrepare's SinkCast()Adrian Prantl1-0/+1
This preserves the debug info for the cast operation in the original location. rdar://problem/33460652 llvm-svn: 317340
2017-11-03re-land [ExpandMemCmp] Split ExpandMemCmp from CodeGen into its own pass."Clement Courbet1-710/+0
Fix undefined references: ExpandMemCmp belongs to CodeGen/, not Scalar/. llvm-svn: 317318
2017-11-02Revert "[ExpandMemCmp] Split ExpandMemCmp from CodeGen into its own pass."Clement Courbet1-0/+710
undefined reference to `llvm::TargetPassConfig::ID' on clang-ppc64le-linux-multistage This reverts commit eea333c33fa73ad225ef28607795984829f65688. llvm-svn: 317213
2017-11-02[ExpandMemCmp] Split ExpandMemCmp from CodeGen into its own pass.Clement Courbet1-710/+0
Summary: This is mostly a noop (most of the test diffs are renamed blocks). There are a few temporary register renames (eax<->ecx) and a few blocks are shuffled around. See the discussion in PR33325 for more details. Reviewers: spatel Subscribers: mgorny Differential Revision: https://reviews.llvm.org/D39456 llvm-svn: 317211
2017-10-31[CGP] Fix the detection of trivial case for addressing modeSerguei Katkov1-10/+9
The address can be presented as a bitcast of baseReg. In this case it is still trivial but OriginalValue != baseReg. llvm-svn: 316980
2017-10-30[CGP] Fix crash on i96 bit multiplyPhilip Reames1-1/+1
Issue found by llvm-isel-fuzzer on OSS fuzz, https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3725 If anyone actually cares about > 64 bit arithmetic, there's a lot more to do in this area. There's a bunch of obviously wrong code in the same function. I don't have the time to fix all of them and am just using this to understand what the workflow for fixing fuzzer cases might look like. llvm-svn: 316967
generated by cgit v1.1 at 2025-10-18 11:39:45 +0000