Handle overflow in dependence analysis lambda ops gracefully

The following tries to handle overflow in the integer computations
done by lambda ops of dependence analysis by failing instead of
silently continuing with overflowed values.

It also avoids treating large unsigned CHREC_RIGHT as negative
unless the chrec is of pointer type and avoids the most negative
integer value to avoid excessive overflow checking (with this
the fix for PR98758 can be partly simplified as seen).

I've added add_hwi and mul_hwi functions computing HOST_WIDE_INT
signed sum and product with indicating overflow, they hopefully
get matched to the appropriate internal functions.

I don't have any testcases triggering overflow in any of the
guarded computations.

2021-01-20  Richard Biener  <rguenther@suse.de>

	* hwint.h (add_hwi): New function.
	(mul_hwi): Likewise.
	* tree-data-ref.c (initialize_matrix_A): Properly translate
	tree constants and avoid HOST_WIDE_INT_MIN.
	(lambda_matrix_row_add): Avoid undefined integer overflow
	and return true on such overflow.
	(lambda_matrix_right_hermite): Handle overflow from
	lambda_matrix_row_add gracefully.  Simplify previous fix.
	(analyze_subscript_affine_affine): Likewise.

1 files changed, 42 insertions, 0 deletions

diff --git a/gcc/hwint.h b/gcc/hwint.h
index 127b013..0e895f8 100644
--- a/gcc/hwint.h
+++ b/gcc/hwint.h
@@ -333,4 +333,46 @@ absu_hwi (HOST_WIDE_INT x)
   return x >= 0 ? (unsigned HOST_WIDE_INT)x : -(unsigned HOST_WIDE_INT)x;
 }
 
+/* Compute the sum of signed A and B and indicate in *OVERFLOW whether
+   that operation overflowed.  */
+
+inline HOST_WIDE_INT
+add_hwi (HOST_WIDE_INT a, HOST_WIDE_INT b, bool *overflow)
+{
+#if GCC_VERSION < 11000
+  unsigned HOST_WIDE_INT result = a + (unsigned HOST_WIDE_INT)b;
+  if ((((result ^ a) & (result ^ b))
+       >> (HOST_BITS_PER_WIDE_INT - 1)) & 1)
+    *overflow = true;
+  else
+    *overflow = false;
+  return result;
+#else
+  HOST_WIDE_INT result;
+  *overflow = __builtin_add_overflow (a, b, &result);
+  return result;
+#endif
+}
+
+/* Compute the product of signed A and B and indicate in *OVERFLOW whether
+   that operation overflowed.  */
+
+inline HOST_WIDE_INT
+mul_hwi (HOST_WIDE_INT a, HOST_WIDE_INT b, bool *overflow)
+{
+#if GCC_VERSION < 11000
+  unsigned HOST_WIDE_INT result = a * (unsigned HOST_WIDE_INT)b;
+  if ((a == -1 && b == HOST_WIDE_INT_MIN)
+      || (a != 0 && (HOST_WIDE_INT)result / a != b))
+    *overflow = true;
+  else
+    *overflow = false;
+  return result;
+#else
+  HOST_WIDE_INT result;
+  *overflow = __builtin_mul_overflow (a, b, &result);
+  return result;
+#endif
+}
+
 #endif /* ! GCC_HWINT_H */