expand: Fix UB in choose_mult_variant [PR105533]

As documented in the function comment, choose_mult_variant attempts to
compute costs of 3 different cases, val, -val and val - 1.
The -val case is actually only done if val fits into host int, so there
should be no overflow, but the val - 1 case is done unconditionally.
val is shwi (but inside of synth_mult already uhwi), so when val is
HOST_WIDE_INT_MIN, val - 1 invokes UB.  The following patch fixes that
by using val - HOST_WIDE_INT_1U, but I'm not really convinced it would
DTRT for > 64-bit modes, so I've guarded it as well.  Though, arch
would need to have really strange costs that something that could be
expressed as x << 63 would be better expressed as (x * 0x7fffffffffffffff) + 1
In the long term, I think we should just rewrite
choose_mult_variant/synth_mult etc. to work on wide_int.

2024-03-07  Jakub Jelinek  <jakub@redhat.com>

	PR middle-end/105533
	* expmed.cc (choose_mult_variant): Only try the val - 1 variant
	if val is not HOST_WIDE_INT_MIN or if mode has exactly
	HOST_BITS_PER_WIDE_INT precision.  Avoid triggering UB while computing
	val - 1.

	* gcc.dg/pr105533.c: New test.

0 files changed, 0 insertions, 0 deletions