| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Rushing this one out before vacation starts. Refactoring on top of
#66505
|
|
This reverts commit 0d0c2298552222b049fa3b8db5efef4b161e51e9.
Includes a bug fix for fcmp one handling, as well as for positive constants.
|
|
These were missed and hopefully avoids assertions when
dc3faf0ed0e3f1ea9e435a006167d9649f865da1 is recommitted.
|
|
This ensures full compare coverage for certain special constants.
|
|
The definition of the pointer of the memory location being queried is
always one such context. Even this conservative guess can be better than
no guess at all in some cases.
Fixes #64666
Co-authored-by: David Goldblatt <davidgoldblatt@meta.com>
|
|
Vectors are always bit-packed and don't respect the elements' alignment
requirements. This is different from arrays. This means offsets of
vector GEPs need to be computed differently than offsets of array GEPs.
This PR fixes many places that rely on an incorrect pattern
that always relies on `DL.getTypeAllocSize(GTI.getIndexedType())`.
We replace these by usages of `GTI.getSequentialElementStride(DL)`,
which is a new helper function added in this PR.
This changes behavior for GEPs into vectors with element types for which
the (bit) size and alloc size is different. This includes two cases:
* Types with a bit size that is not a multiple of a byte, e.g. i1.
GEPs into such vectors are questionable to begin with, as some elements
are not even addressable.
* Overaligned types, e.g. i16 with 32-bit alignment.
Existing tests are unaffected, but a miscompilation of a new test is fixed.
---------
Co-authored-by: Nikita Popov <github@npopov.com>
|
|
This patch removes redundant SPF support
(https://github.com/llvm/llvm-project/commit/5350e1b5096aa4707aa525baf7398d93b4a4f1a5)
from `computeKnownBitsFromOperator` as we always canonicalize a SPF into
an intrinsic call.
Compile-time improvement:
http://llvm-compile-time-tracker.com/compare.php?from=3dc0638cfc19e140daff7bf1281648daca8212fa&to=8771ef0749fb2ba4304dc68d418c88ec5769346f&stat=instructions:u
|stage1-O3|stage1-ReleaseThinLTO|stage1-ReleaseLTO-g|stage1-O0-g|stage2-O3|stage2-O0-g|stage2-clang|
|--|--|--|--|--|--|--|
-0.01%|-0.01%|+0.01%|+0.00%|+0.01%|+0.04%|-0.01%|
|
|
This patch tries to fold minmax intrinsic by using
`computeConstantRangeIncludingKnownBits`.
Fixes regression in
[_karatsuba_rec:cpython/Modules/_decimal/libmpdec/mpdecimal.c](https://github.com/python/cpython/blob/c31943af16f885c8cf5d5a690c25c366afdb2862/Modules/_decimal/libmpdec/mpdecimal.c#L5460-L5462),
which was introduced by #71396.
See also
https://github.com/dtcxzyw/llvm-opt-benchmark/issues/16#issuecomment-1865875756.
Alive2 for splat vectors with undef: https://alive2.llvm.org/ce/z/J8hKWd
|
|
Currently isGuaranteedNotToBeUndef() is the same as
isGuaranteedNotToBeUndefOrPoison(). This function is used in places
where we only care about undef (due to multi-use issues), not poison.
Make it more precise by only considering instructions that can create
undef (like loads or call), and ignore those that can only create
poison. In particular, we can ignore poison-generating flags.
This means that inferring more flags has less chance to pessimize other
transforms.
|
|
Shufflevector semantics have changed so that poison mask elements
return poison rather than undef. Reflect this in the
canCreateUndefOrPoison() implementation.
|
|
If all the demanded elements are poison, return unknown instead of
conflict to avoid downstream assertions.
Fixes https://github.com/llvm/llvm-project/issues/75505.
|
|
When the sub arguments are ptr2int it is not possible to determine
computeKnownBits() of its arguments.
For scalar case generally sub of 2 ptr2int are converted to sub of
indexes.
However a loop with recursive GEP/PHI where the arguments to sub is of
type ptr2int, if it is possible to determine that a sub of this GEP and
another pointer with the same base is KnownNonZero we can return this.
This helps subsequent passes to optimize the loop further.
|
|
Don't bother computing known bits for the second operand if we
know nothing about the first.
|
|
(x - y) != 0 is true iff x != y, so use the isKnownNonEqual()
helper, which knows some additional tricks.
|
|
Handles canonical icmp eq(ptr1, ptr2) -> where ptr1/ptr2 is a recursive
GEP.
Can helps scenarios where InstCombineCompares folds icmp eq(sub(ptr2int,
ptr2int), 0) -> icmp eq(ptr1, ptr2)
and
icmp eq(phi(sub(ptr2int, ptr2int), ...)) -> phi i1 (icmp eq(sub(ptr2int,
ptr2int), 0), ....)
|
|
Alive2: https://alive2.llvm.org/ce/z/kiGxCf
Fixes #70374.
|
|
The returned attribute can be used when it is possible to
"losslessly bitcast" between the argument and return type,
including between two vector types.
computeKnownBits() would crash in this case, isKnownNonZero()
would potentially produce a miscompile.
Fixes https://github.com/llvm/llvm-project/issues/74722.
|
|
|
|
matchSimpleRecurrence. (#74678)
Operator allows the phi operand to be a ConstantExpr. A ConstantExpr is
a valid operand to a phi, but is never going to be a recurrence.
We can only match a BinaryOperator so use that instead.
|
|
I'm not sure whether it's possible to cause a miscompile due to
the missing check right now, as the affected values mechanism
effectively protects us against this. This becomes a problem for
an upcoming patch though.
|
|
This adds support for using dominating conditions in computeKnownBits()
when called from InstCombine. The implementation uses a
DomConditionCache, which stores which branches may provide information
that is relevant for a given value.
DomConditionCache is similar to AssumptionCache, but does not try to do
any kind of automatic tracking. Relevant branches have to be explicitly
registered and invalidated values explicitly removed. The necessary
tracking is done inside InstCombine.
The reason why this doesn't just do exactly the same thing as
AssumptionCache is that a lot more transforms touch branches and branch
conditions than assumptions. AssumptionCache is an immutable analysis
and mostly gets away with this because only a handful of places have to
register additional assumptions (mostly as a result of cloning). This is
very much not the case for branches.
This change regresses compile-time by about ~0.2%. It also improves
stage2-O0-g builds by about ~0.2%, which indicates that this change results
in additional optimizations inside clang itself.
Fixes https://github.com/llvm/llvm-project/issues/74242.
|
|
haveNoCommonBitsSetSpecialCases. (#74390)
It's not safe for InstCombine to add disjoint metadata when converting
Add to Or otherwise.
I've added noundef attribute to preserve existing test behavior.
|
|
(#66505)""
This reverts commit d55692d60d218f402ce107520daabed15f2d9ef6.
See discussion in #66505: assertion fires in OSS build of TensorFlow.
|
|
This should say "Assume that VL output is <= 65536".
|
|
We have a bunch of places where we have to guard against undef
to avoid multi-use issues, but would be fine with poison. Use a
different function for these to make it clear, and to indicate that
this check can be removed once we no longer support undef. I've
replaced some of the obvious cases, but there's probably more.
For now, the implementation is the same as UndefOrPoison, it just
has a more precise name.
|
|
Extract a function and call it with both operand orders, so that
we don't have to explicitly commute every single pattern.
|
|
If both icmps have the same operands and the RHS is constant, we
would currently go into the isImpliedCondMatchingOperands() code
path, instead of the isImpliedCondCommonOperandWithConstants()
path. Both are correct, but the latter can produce more accurate
results if the implication is dependent on the sign.
|
|
This reverts commit 96a0d714d58e48c363ee6abbbcdfd7a6ce646ac1.
Avoid assert with dynamic denormal-fp-math We don't recognize compares
with 0 as an exact class test if we don't know the denormal mode. We could
try to do better here, but it's probably not worth it.
Fixes asserts reported after 1adce7d8e47e2438f99f91607760b825e5e3cc37
|
|
(#74021)
Similar to div, the rem should also proof its second operand is
non-zero, otherwise it is a UB.
Fix https://github.com/llvm/llvm-project/issues/71782
|
|
It already used it internally, make the public API use it as well.
|
|
|
|
It looks like this function is actually unused.
|
|
|
|
|
|
In preparation for handling non-assume context-sensitive facts.
|
|
Call computeKnownBits() with SimplifyQuery to make sure it gets
all available analyses, even if more are added in the future.
As this code is performance-critical, I'm exporting the variant
with by-ref KnownBits and SimplifyQuery, as the variant returning
KnownBits is measurably slower in this context.
|
|
If you need this, use one of the variants returning KnownBits
instead.
|
|
This will alllow using it with an inverted predicate in the future.
|
|
For most practical purposes, the only KnownBits patterns we care about are
those involving a constant comparison RHS and constant mask. However,
the actual implementation is written in a very general way -- and of
course, with basically no test coverage of those generalizations.
This patch reduces the implementation to only handle cases with constant
operands. The test changes are all in "make sure we don't crash" tests.
The motivation for this change is an upcoming patch to handling dominating
conditions in computeKnownBits(). Handling non-constant RHS would add
significant additional compile-time overhead in that case, without any
significant impact on optimization quality.
|
|
We are already iterating over all assumes in AC, so handle
operand bundle based assumes in the same loop, instead of querying
them separately.
To keep the debug counter working, make it work per-bundle rather
than per-value.
|
|
This patch handles `poison` elements of non-splat vectors in
`computeKnownBits`. It addresses test changes after I delete the
duplicate logic in https://github.com/llvm/llvm-project/pull/72535.
See also @nikic's comment:
https://github.com/llvm/llvm-project/pull/72535#pullrequestreview-1736991557
|
|
For all practical purposes, we only care about comparisons with
constant RHS in this code. In that case, an invert will be
canonicalized into the constant and it will be handled by other cases.
Given the complete lack of test coverage, I'm removing this code.
|
|
Related patch: https://github.com/llvm/llvm-project/pull/68331
This missed optimization is discovered with the help of
https://github.com/AliveToolkit/alive2/pull/962.
|
|
This apparently shows up somewhere in chromium. We also are missing a
canonicalization to an equality compare with inf.
|
|
This causes asserts to fire:
llvm/lib/Analysis/ValueTracking.cpp:4262:
std::tuple<Value *, FPClassTest, FPClassTest> llvm::fcmpImpliesClass(CmpInst::Predicate, const Function &, Value *, const APFloat *, bool):
Assertion `(RHSClass == fcPosNormal || RHSClass == fcNegNormal || RHSClass == fcPosSubnormal || RHSClass == fcNegSubnormal) && "should have been recognized as an exact class test"' failed.
See comments on the PR.
> Previously we could recognize exact class tests performed by
> an fcmp with special values (0s, infs and smallest normal).
> Expand this to recognize the implied classes by a compare with a general
> constant. e.g. fcmp ogt x, 1 implies positive and non-0.
>
> The API should be better merged with fcmpToClassTest but that
> made the diff way bigger, will try to do that in a future
> patch.
This reverts commit dc3faf0ed0e3f1ea9e435a006167d9649f865da1.
|
|
Previously we could recognize exact class tests performed by
an fcmp with special values (0s, infs and smallest normal).
Expand this to recognize the implied classes by a compare with a general
constant. e.g. fcmp ogt x, 1 implies positive and non-0.
The API should be better merged with fcmpToClassTest but that
made the diff way bigger, will try to do that in a future
patch.
|
|
zext nneg was recently added to the IR in #67982. This patch teaches
demanded bits and known bits about the semantics of the instruction, and
adds a couple of test cases to illustrate basic functionality.
|
|
Use the constant folding API instead.
|
|
Use ConstantFoldIntegerCast() instead, to remove the reliance on
constant expressions.
|
|
Basic way to recursively analyze `select` in `isKnownNonEqual`: `select
%c, %t, %f` is non-equal to `%x` if `%t` is non-equal to `%x` and `%f`
is non-equal to `%x`.
|
