lower-bitint: Fix up lower_addsub_overflow [PR112807]

lower_addsub_overflow uses handle_cast or handle_operand to extract current
limb from the operands.  Both of those functions heavily assume that they
return a large or huge BITINT_TYPE.  The problem in the testcase is that
this is violated.  Normally, lower_addsub_overflow isn't even called if
neither the return's type element type nor any of the operand is large/huge
BITINT_TYPE (on x86_64 129+ bits), for middle BITINT_TYPE (on x86_64 65-128
bits) some other code casts such operands to {,unsigned }__int128.
In the testcase the result is complex unsigned, so small, but one of the
arguments is _BitInt(256), so lower_addsub_overflow is called.  But
range_for_prec asks the ranger for ranges of the operands and in this
case the first argument has [0, 0xffffffff] range and second [-2, 1], so
unsigned 32-bit and signed 2-bit, and in such case the code for
handle_operand/handle_cast purposes would use the _BitInt(256) type for the
first operand (ok), but because prec3 aka maximum of result precision and
the VRP computes ranges of the arguments is 32, use cast to 32-bit
BITINT_TYPE, which is why it didn't work correctly.
The following patch ensures that in such cases we use handle_cast to the
type of the other argument.

Perhaps incrementally, we could try to optimize this in an earlier phase,
see that while the .{ADD,SUB}_OVERFLOW has large/huge _BitInt argument, as
ranger says it fits into a smaller type, add a cast of the larger argument
to the smaller precision type in which it fits.  Either in
gimple_lower_bitint, or match.pd.  An argument for the latter is that e.g.
complex unsigned .ADD_OVERFLOW (unsigned_long_long_arg, unsigned_arg)
where ranger says unsigned_long_long_arg fits into unsigned 32-bit could
be also more efficient as
.ADD_OVERFLOW ((unsigned) unsigned_long_long_arg, unsigned_arg)

2023-12-03  Jakub Jelinek  <jakub@redhat.com>

	PR middle-end/112807
	* gimple-lower-bitint.cc (bitint_large_huge::lower_addsub_overflow):
	When choosing type0 and type1 types, if prec3 has small/middle bitint
	kind, use maximum of type0 and type1's precision instead of prec3.

	* gcc.dg/bitint-46.c: New test.

0 files changed, 0 insertions, 0 deletions