df: Fix up handling of paradoxical subregs in debug insns [PR101170]

The recent addition of gcc_assert (regno < endregno); triggers during
glibc build on m68k.
The problem is that RA decisions shouldn't depend on expressions in
DEBUG_INSNs and those expressions can contain paradoxical subregs of certain
pseudos.  If RA then decides to allocate the pseudo to a register
with very small hard register REGNO, we can trigger the new assert,
as (int) subreg_regno_offset may be negative on big endian and the small
REGNO + the negative offset can wrap around.

The following patch in that case records the range from the REGNO 0 to
endregno, before the addition of the assert as both regno and endregno are
unsigned it wouldn't record anything at all silently.

2021-06-24  Jakub Jelinek  <jakub@redhat.com>

	PR middle-end/101170
	* df-scan.c (df_ref_record): For paradoxical big-endian SUBREGs
	where regno + subreg_regno_offset wraps around use 0 as starting
	regno.

	* gcc.dg/pr101170.c: New test.

2 files changed, 52 insertions, 3 deletions

diff --git a/gcc/df-scan.c b/gcc/df-scan.c
index e9da64f..3dbda7a 100644
--- a/gcc/df-scan.c
+++ b/gcc/df-scan.c
@@ -2576,9 +2576,21 @@ df_ref_record (enum df_ref_class cl,
 
       if (GET_CODE (reg) == SUBREG)
 	{
-	  regno += subreg_regno_offset (regno, GET_MODE (SUBREG_REG (reg)),
-					SUBREG_BYTE (reg), GET_MODE (reg));
-	  endregno = regno + subreg_nregs (reg);
+	  int off = subreg_regno_offset (regno, GET_MODE (SUBREG_REG (reg)),
+					 SUBREG_BYTE (reg), GET_MODE (reg));
+	  unsigned int nregno = regno + off;
+	  endregno = nregno + subreg_nregs (reg);
+	  if (off < 0 && regno < (unsigned) -off)
+	    /* Deal with paradoxical SUBREGs on big endian where
+	       in debug insns the hard reg number might be smaller
+	       than -off, such as (subreg:DI (reg:SI 0 [+4 ]) 0));
+	       RA decisions shouldn't be affected by debug insns
+	       and so RA can decide to put pseudo into a hard reg
+	       with small REGNO, even when it is referenced in
+	       a paradoxical SUBREG in a debug insn.  */
+	    regno = 0;
+	  else
+	    regno = nregno;
 	}
       else
 	endregno = END_REGNO (reg);
diff --git a/gcc/testsuite/gcc.dg/pr101170.c b/gcc/testsuite/gcc.dg/pr101170.c
new file mode 100644
index 0000000..fc81062
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/pr101170.c
@@ -0,0 +1,37 @@
+/* PR middle-end/101170 */
+/* { dg-do compile } */
+/* { dg-options "-O2 -g" } */
+
+#include <stdarg.h>
+
+struct S { int a; int b[4]; } s;
+va_list ap;
+int i;
+long long l;
+
+struct S
+foo (int x)
+{
+  struct S a = {};
+  do
+    if (x)
+      return a;
+  while (1);
+}
+
+__attribute__((noipa)) void
+bar (void)
+{
+  for (; i; i++)
+    l |= va_arg (ap, long long) << s.b[i];
+  if (l)
+    foo (l);
+}
+
+void
+baz (int v, ...)
+{
+  va_start (ap, v);
+  bar ();
+  va_end (ap);
+}