i386: Fix ICEs with SUBREGs from vector etc. constants to XFmode [PR114184]

The Intel extended format has the various weird number categories,
pseudo denormals, pseudo infinities, pseudo NaNs and unnormals.
Those are not representable in the GCC real_value and so neither
GIMPLE nor RTX VIEW_CONVERT_EXPR/SUBREG folding folds those into
constants.

As can be seen on the following testcase, because it isn't folded
(since GCC 12, before that we were folding it) we can end up with
a SUBREG of a CONST_VECTOR or similar constant, which isn't valid
general_operand, so we ICE during vregs pass trying to recognize
the move instruction.
Initially I thought it is a middle-end bug, the movxf instruction
has general_operand predicate, but the middle-end certainly never
tests that predicate, seems moves are special optabs.
And looking at other mov optabs, e.g. for vector modes the i386
patterns use nonimmediate_operand predicate on the input, yet
ix86_expand_vector_move deals with CONSTANT_P and SUBREG of CONSTANT_P
arguments which if the predicate was checked couldn't ever make it through.

The following patch handles this case similarly to the
ix86_expand_vector_move's SUBREG of CONSTANT_P case, does it just for XFmode
because I believe that is the only mode that needs it from the scalar ones,
others should just be folded.

2024-03-04  Jakub Jelinek  <jakub@redhat.com>

	PR target/114184
	* config/i386/i386-expand.cc (ix86_expand_move): If XFmode op1
	is SUBREG of CONSTANT_P, force the SUBREG_REG into memory or
	register.

	* gcc.target/i386/pr114184.c: New test.

(cherry picked from commit ea1c16f95b8fbaba4a7f3663ff9933ebedfb92a5)

2 files changed, 39 insertions, 0 deletions

diff --git a/gcc/config/i386/i386-expand.cc b/gcc/config/i386/i386-expand.cc
index 304d5e7..c57a8f5 100644
--- a/gcc/config/i386/i386-expand.cc
+++ b/gcc/config/i386/i386-expand.cc
@@ -350,6 +350,23 @@ ix86_expand_move (machine_mode mode, rtx operands[])
 
     default:
       break;
+
+    case SUBREG:
+      /* As not all values in XFmode are representable in real_value,
+	 we might be called with unfoldable SUBREGs of constants.  */
+      if (mode == XFmode
+	  && CONSTANT_P (SUBREG_REG (op1))
+	  && can_create_pseudo_p ())
+	{
+	  machine_mode imode = GET_MODE (SUBREG_REG (op1));
+	  rtx r = force_const_mem (imode, SUBREG_REG (op1));
+	  if (r)
+	    r = validize_mem (r);
+	  else
+	    r = force_reg (imode, SUBREG_REG (op1));
+	  op1 = simplify_gen_subreg (mode, r, imode, SUBREG_BYTE (op1));
+	}
+      break;
     }
 
   if ((flag_pic || MACHOPIC_INDIRECT)
diff --git a/gcc/testsuite/gcc.target/i386/pr114184.c b/gcc/testsuite/gcc.target/i386/pr114184.c
new file mode 100644
index 0000000..360b3b9
--- /dev/null
+++ b/gcc/testsuite/gcc.target/i386/pr114184.c
@@ -0,0 +1,22 @@
+/* PR target/114184 */
+/* { dg-do compile } */
+/* { dg-options "-Og -mavx2" } */
+
+typedef unsigned char V __attribute__((vector_size (32)));
+typedef unsigned char W __attribute__((vector_size (16)));
+
+_Complex long double
+foo (void)
+{
+  _Complex long double d;
+  *(V *)&d = (V) { 149, 136, 89, 42, 38, 240, 196, 194 };
+  return d;
+}
+
+long double
+bar (void)
+{
+  long double d;
+  *(W *)&d = (W) { 149, 136, 89, 42, 38, 240, 196, 194 };
+  return d;
+}