poly_int: GET_MODE_NUNITS

This patch changes GET_MODE_NUNITS from unsigned char
to poly_uint16, although it remains a macro when compiling
target code with NUM_POLY_INT_COEFFS == 1.

We can handle permuted loads and stores for variable nunits if
the number of statements is a power of 2, but not otherwise.

The to_constant call in make_vector_type goes away in a later patch.

2018-01-03  Richard Sandiford  <richard.sandiford@linaro.org>
	    Alan Hayward  <alan.hayward@arm.com>
	    David Sherwood  <david.sherwood@arm.com>

gcc/
	* machmode.h (mode_nunits): Change from unsigned char to
	poly_uint16_pod.
	(ONLY_FIXED_SIZE_MODES): New macro.
	(pod_mode::measurement_type, scalar_int_mode::measurement_type)
	(scalar_float_mode::measurement_type, scalar_mode::measurement_type)
	(complex_mode::measurement_type, fixed_size_mode::measurement_type):
	New typedefs.
	(mode_to_nunits): Return a poly_uint16 rather than an unsigned short.
	(GET_MODE_NUNITS): Return a constant if ONLY_FIXED_SIZE_MODES,
	or if measurement_type is not polynomial.
	* genmodes.c (ZERO_COEFFS): New macro.
	(emit_mode_nunits_inline): Make mode_nunits_inline return a
	poly_uint16.
	(emit_mode_nunits): Change the type of mode_nunits to poly_uint16_pod.
	Use ZERO_COEFFS when emitting initializers.
	* data-streamer.h (bp_pack_poly_value): New function.
	(bp_unpack_poly_value): Likewise.
	* lto-streamer-in.c (lto_input_mode_table): Use bp_unpack_poly_value
	for GET_MODE_NUNITS.
	* lto-streamer-out.c (lto_write_mode_table): Use bp_pack_poly_value
	for GET_MODE_NUNITS.
	* tree.c (make_vector_type): Remove temporary shim and make
	the real function take the number of units as a poly_uint64
	rather than an int.
	(build_vector_type_for_mode): Handle polynomial nunits.
	* dwarf2out.c (loc_descriptor, add_const_value_attribute): Likewise.
	* emit-rtl.c (const_vec_series_p_1): Likewise.
	(gen_rtx_CONST_VECTOR): Likewise.
	* fold-const.c (test_vec_duplicate_folding): Likewise.
	* genrecog.c (validate_pattern): Likewise.
	* optabs-query.c (can_vec_perm_var_p, can_mult_highpart_p): Likewise.
	* optabs-tree.c (expand_vec_cond_expr_p): Likewise.
	* optabs.c (expand_vector_broadcast, expand_binop_directly): Likewise.
	(shift_amt_for_vec_perm_mask, expand_vec_perm_var): Likewise.
	(expand_vec_cond_expr, expand_mult_highpart): Likewise.
	* rtlanal.c (subreg_get_info): Likewise.
	* tree-vect-data-refs.c (vect_grouped_store_supported): Likewise.
	(vect_grouped_load_supported): Likewise.
	* tree-vect-generic.c (type_for_widest_vector_mode): Likewise.
	* tree-vect-loop.c (have_whole_vector_shift): Likewise.
	* simplify-rtx.c (simplify_unary_operation_1): Likewise.
	(simplify_const_unary_operation, simplify_binary_operation_1)
	(simplify_const_binary_operation, simplify_ternary_operation)
	(test_vector_ops_duplicate, test_vector_ops): Likewise.
	(simplify_immed_subreg): Use GET_MODE_NUNITS on a fixed_size_mode
	instead of CONST_VECTOR_NUNITS.
	* varasm.c (output_constant_pool_2): Likewise.
	* rtx-vector-builder.c (rtx_vector_builder::build): Only include the
	explicit-encoded elements in the XVEC for variable-length vectors.

gcc/ada/
	* gcc-interface/misc.c (enumerate_modes): Handle polynomial
	GET_MODE_NUNITS.

Co-Authored-By: Alan Hayward <alan.hayward@arm.com>
Co-Authored-By: David Sherwood <david.sherwood@arm.com>

From-SVN: r256195

23 files changed, 325 insertions, 135 deletions

diff --git a/gcc/ChangeLog b/gcc/ChangeLog
index 5608253..7a96624 100644
--- a/gcc/ChangeLog
+++ b/gcc/ChangeLog
@@ -1,4 +1,58 @@
 2018-01-03  Richard Sandiford  <richard.sandiford@linaro.org>
+	    Alan Hayward  <alan.hayward@arm.com>
+	    David Sherwood  <david.sherwood@arm.com>
+
+	* machmode.h (mode_nunits): Change from unsigned char to
+	poly_uint16_pod.
+	(ONLY_FIXED_SIZE_MODES): New macro.
+	(pod_mode::measurement_type, scalar_int_mode::measurement_type)
+	(scalar_float_mode::measurement_type, scalar_mode::measurement_type)
+	(complex_mode::measurement_type, fixed_size_mode::measurement_type):
+	New typedefs.
+	(mode_to_nunits): Return a poly_uint16 rather than an unsigned short.
+	(GET_MODE_NUNITS): Return a constant if ONLY_FIXED_SIZE_MODES,
+	or if measurement_type is not polynomial.
+	* genmodes.c (ZERO_COEFFS): New macro.
+	(emit_mode_nunits_inline): Make mode_nunits_inline return a
+	poly_uint16.
+	(emit_mode_nunits): Change the type of mode_nunits to poly_uint16_pod.
+	Use ZERO_COEFFS when emitting initializers.
+	* data-streamer.h (bp_pack_poly_value): New function.
+	(bp_unpack_poly_value): Likewise.
+	* lto-streamer-in.c (lto_input_mode_table): Use bp_unpack_poly_value
+	for GET_MODE_NUNITS.
+	* lto-streamer-out.c (lto_write_mode_table): Use bp_pack_poly_value
+	for GET_MODE_NUNITS.
+	* tree.c (make_vector_type): Remove temporary shim and make
+	the real function take the number of units as a poly_uint64
+	rather than an int.
+	(build_vector_type_for_mode): Handle polynomial nunits.
+	* dwarf2out.c (loc_descriptor, add_const_value_attribute): Likewise.
+	* emit-rtl.c (const_vec_series_p_1): Likewise.
+	(gen_rtx_CONST_VECTOR): Likewise.
+	* fold-const.c (test_vec_duplicate_folding): Likewise.
+	* genrecog.c (validate_pattern): Likewise.
+	* optabs-query.c (can_vec_perm_var_p, can_mult_highpart_p): Likewise.
+	* optabs-tree.c (expand_vec_cond_expr_p): Likewise.
+	* optabs.c (expand_vector_broadcast, expand_binop_directly): Likewise.
+	(shift_amt_for_vec_perm_mask, expand_vec_perm_var): Likewise.
+	(expand_vec_cond_expr, expand_mult_highpart): Likewise.
+	* rtlanal.c (subreg_get_info): Likewise.
+	* tree-vect-data-refs.c (vect_grouped_store_supported): Likewise.
+	(vect_grouped_load_supported): Likewise.
+	* tree-vect-generic.c (type_for_widest_vector_mode): Likewise.
+	* tree-vect-loop.c (have_whole_vector_shift): Likewise.
+	* simplify-rtx.c (simplify_unary_operation_1): Likewise.
+	(simplify_const_unary_operation, simplify_binary_operation_1)
+	(simplify_const_binary_operation, simplify_ternary_operation)
+	(test_vector_ops_duplicate, test_vector_ops): Likewise.
+	(simplify_immed_subreg): Use GET_MODE_NUNITS on a fixed_size_mode
+	instead of CONST_VECTOR_NUNITS.
+	* varasm.c (output_constant_pool_2): Likewise.
+	* rtx-vector-builder.c (rtx_vector_builder::build): Only include the
+	explicit-encoded elements in the XVEC for variable-length vectors.
+
+2018-01-03  Richard Sandiford  <richard.sandiford@linaro.org>
 
 	* lra-constraints.c (curr_insn_transform): Use partial_subreg_p.
 
diff --git a/gcc/ada/ChangeLog b/gcc/ada/ChangeLog
index 253bcff..a4f1af0 100644
--- a/gcc/ada/ChangeLog
+++ b/gcc/ada/ChangeLog
@@ -1,3 +1,10 @@
+2018-01-03  Richard Sandiford  <richard.sandiford@linaro.org>
+	    Alan Hayward  <alan.hayward@arm.com>
+	    David Sherwood  <david.sherwood@arm.com>
+
+	* gcc-interface/misc.c (enumerate_modes): Handle polynomial
+	GET_MODE_NUNITS.
+
 2018-01-03  Jakub Jelinek  <jakub@redhat.com>
 
 	Update copyright years.
diff --git a/gcc/ada/gcc-interface/misc.c b/gcc/ada/gcc-interface/misc.c
index 7401d2d..48add09 100644
--- a/gcc/ada/gcc-interface/misc.c
+++ b/gcc/ada/gcc-interface/misc.c
@@ -1301,9 +1301,10 @@ enumerate_modes (void (*f) (const char *, int, int, int, int, int, int, int))
 	  }
 
       /* If no predefined C types were found, register the mode itself.  */
-      if (!skip_p)
+      int nunits;
+      if (!skip_p && GET_MODE_NUNITS (i).is_constant (&nunits))
 	f (GET_MODE_NAME (i), digs, complex_p,
-	   vector_p ? GET_MODE_NUNITS (i) : 0, float_rep,
+	   vector_p ? nunits : 0, float_rep,
 	   GET_MODE_PRECISION (i), GET_MODE_BITSIZE (i),
 	   GET_MODE_ALIGNMENT (i));
     }
diff --git a/gcc/data-streamer.h b/gcc/data-streamer.h
index 0ed6e36..3f969d5 100644
--- a/gcc/data-streamer.h
+++ b/gcc/data-streamer.h
@@ -126,6 +126,17 @@ bp_pack_value (struct bitpack_d *bp, bitpack_word_t val, unsigned nbits)
   bp->pos = pos;
 }
 
+/* Pack VAL into the bit-packing context BP, using NBITS for each
+   coefficient.  */
+static inline void
+bp_pack_poly_value (struct bitpack_d *bp,
+		    const poly_int<NUM_POLY_INT_COEFFS, bitpack_word_t> &val,
+		    unsigned nbits)
+{
+  for (int i = 0; i < NUM_POLY_INT_COEFFS; ++i)
+    bp_pack_value (bp, val.coeffs[i], nbits);
+}
+
 /* Finishes bit-packing of BP.  */
 static inline void
 streamer_write_bitpack (struct bitpack_d *bp)
@@ -174,6 +185,17 @@ bp_unpack_value (struct bitpack_d *bp, unsigned nbits)
   return val & mask;
 }
 
+/* Unpacks a polynomial value from the bit-packing context BP in which each
+   coefficient has NBITS bits.  */
+static inline poly_int<NUM_POLY_INT_COEFFS, bitpack_word_t>
+bp_unpack_poly_value (struct bitpack_d *bp, unsigned nbits)
+{
+  poly_int_pod<NUM_POLY_INT_COEFFS, bitpack_word_t> x;
+  for (int i = 0; i < NUM_POLY_INT_COEFFS; ++i)
+    x.coeffs[i] = bp_unpack_value (bp, nbits);
+  return x;
+}
+
 
 /* Write a character to the output block.  */
 
diff --git a/gcc/dwarf2out.c b/gcc/dwarf2out.c
index b3268cc..2b6e588 100644
--- a/gcc/dwarf2out.c
+++ b/gcc/dwarf2out.c
@@ -16010,8 +16010,11 @@ loc_descriptor (rtx rtl, machine_mode mode,
 
       if (mode != VOIDmode && (dwarf_version >= 4 || !dwarf_strict))
 	{
+	  unsigned int length;
+	  if (!CONST_VECTOR_NUNITS (rtl).is_constant (&length))
+	    return NULL;
+
 	  unsigned int elt_size = GET_MODE_UNIT_SIZE (GET_MODE (rtl));
-	  unsigned int length = CONST_VECTOR_NUNITS (rtl);
 	  unsigned char *array
 	    = ggc_vec_alloc<unsigned char> (length * elt_size);
 	  unsigned int i;
@@ -18817,9 +18820,12 @@ add_const_value_attribute (dw_die_ref die, rtx rtl)
 
     case CONST_VECTOR:
       {
+	unsigned int length;
+	if (!CONST_VECTOR_NUNITS (rtl).is_constant (&length))
+	  return false;
+
 	machine_mode mode = GET_MODE (rtl);
 	unsigned int elt_size = GET_MODE_UNIT_SIZE (mode);
-	unsigned int length = CONST_VECTOR_NUNITS (rtl);
 	unsigned char *array
 	  = ggc_vec_alloc<unsigned char> (length * elt_size);
 	unsigned int i;
diff --git a/gcc/emit-rtl.c b/gcc/emit-rtl.c
index 0c81493..626567a 100644
--- a/gcc/emit-rtl.c
+++ b/gcc/emit-rtl.c
@@ -5987,7 +5987,7 @@ const_vec_series_p_1 (const_rtx x, rtx *base_out, rtx *step_out)
   /* A non-duplicated vector with two elements can always be seen as a
      series with a nonzero step.  Longer vectors must have a stepped
      encoding.  */
-  if (CONST_VECTOR_NUNITS (x) != 2
+  if (maybe_ne (CONST_VECTOR_NUNITS (x), 2)
       && !CONST_VECTOR_STEPPED_P (x))
     return false;
 
@@ -6068,7 +6068,7 @@ gen_const_vector (machine_mode mode, int constant)
 rtx
 gen_rtx_CONST_VECTOR (machine_mode mode, rtvec v)
 {
-  gcc_assert (GET_MODE_NUNITS (mode) == GET_NUM_ELEM (v));
+  gcc_assert (known_eq (GET_MODE_NUNITS (mode), GET_NUM_ELEM (v)));
 
   /* If the values are all the same, check to see if we can use one of the
      standard constant vectors.  */
diff --git a/gcc/fold-const.c b/gcc/fold-const.c
index 9f8d76e..9f558e2 100644
--- a/gcc/fold-const.c
+++ b/gcc/fold-const.c
@@ -14625,7 +14625,7 @@ test_vec_duplicate_folding ()
   scalar_int_mode int_mode = SCALAR_INT_TYPE_MODE (ssizetype);
   machine_mode vec_mode = targetm.vectorize.preferred_simd_mode (int_mode);
   /* This will be 1 if VEC_MODE isn't a vector mode.  */
-  unsigned int nunits = GET_MODE_NUNITS (vec_mode);
+  poly_uint64 nunits = GET_MODE_NUNITS (vec_mode);
 
   tree type = build_vector_type (ssizetype, nunits);
   tree dup5_expr = fold_unary (VEC_DUPLICATE_EXPR, type, ssize_int (5));
diff --git a/gcc/genmodes.c b/gcc/genmodes.c
index ce0046c..efcceb3 100644
--- a/gcc/genmodes.c
+++ b/gcc/genmodes.c
@@ -901,6 +901,16 @@ calc_wider_mode (void)
     }
 }
 
+/* Text to add to the constant part of a poly_int_pod initializer in
+   order to fill out te whole structure.  */
+#if NUM_POLY_INT_COEFFS == 1
+#define ZERO_COEFFS ""
+#elif NUM_POLY_INT_COEFFS == 2
+#define ZERO_COEFFS ", 0"
+#else
+#error "Unknown value of NUM_POLY_INT_COEFFS"
+#endif
+
 /* Output routines.  */
 
 #define tagged_printf(FMT, ARG, TAG) do {		\
@@ -1008,11 +1018,10 @@ inline __attribute__((__always_inline__))\n\
 #else\n\
 extern __inline__ __attribute__((__always_inline__, __gnu_inline__))\n\
 #endif\n\
-unsigned char\n\
+poly_uint16\n\
 mode_nunits_inline (machine_mode mode)\n\
 {\n\
-  extern const unsigned char mode_nunits[NUM_MACHINE_MODES];\n\
-  gcc_assert (mode >= 0 && mode < NUM_MACHINE_MODES);\n\
+  extern poly_uint16_pod mode_nunits[NUM_MACHINE_MODES];\n\
   switch (mode)\n\
     {");
 
@@ -1381,10 +1390,10 @@ emit_mode_nunits (void)
   int c;
   struct mode_data *m;
 
-  print_decl ("unsigned char", "mode_nunits", "NUM_MACHINE_MODES");
+  print_decl ("poly_uint16_pod", "mode_nunits", "NUM_MACHINE_MODES");
 
   for_all_modes (c, m)
-    tagged_printf ("%u", m->ncomponents, m->name);
+    tagged_printf ("{ %u" ZERO_COEFFS " }", m->ncomponents, m->name);
 
   print_closer ();
 }
diff --git a/gcc/genrecog.c b/gcc/genrecog.c
index 31bf0fe..663df8c 100644
--- a/gcc/genrecog.c
+++ b/gcc/genrecog.c
@@ -746,14 +746,20 @@ validate_pattern (rtx pattern, md_rtx_info *info, rtx set, int set_code)
 	    = VECTOR_MODE_P (mode) ? GET_MODE_INNER (mode) : mode;
 	  if (GET_CODE (XEXP (pattern, 1)) == PARALLEL)
 	    {
-	      int expected = VECTOR_MODE_P (mode) ? GET_MODE_NUNITS (mode) : 1;
-	      if (XVECLEN (XEXP (pattern, 1), 0) != expected)
+	      int expected = 1;
+	      unsigned int nelems;
+	      if (VECTOR_MODE_P (mode)
+		  && !GET_MODE_NUNITS (mode).is_constant (&expected))
+		error_at (info->loc,
+			  "vec_select with variable-sized mode %s",
+			  GET_MODE_NAME (mode));
+	      else if (XVECLEN (XEXP (pattern, 1), 0) != expected)
 		error_at (info->loc,
 			  "vec_select parallel with %d elements, expected %d",
 			  XVECLEN (XEXP (pattern, 1), 0), expected);
-	      else if (VECTOR_MODE_P (imode))
+	      else if (VECTOR_MODE_P (imode)
+		       && GET_MODE_NUNITS (imode).is_constant (&nelems))
 		{
-		  unsigned int nelems = GET_MODE_NUNITS (imode);
 		  int i;
 		  for (i = 0; i < expected; ++i)
 		    if (CONST_INT_P (XVECEXP (XEXP (pattern, 1), 0, i))
diff --git a/gcc/lto-streamer-in.c b/gcc/lto-streamer-in.c
index d2274ce..80b5671 100644
--- a/gcc/lto-streamer-in.c
+++ b/gcc/lto-streamer-in.c
@@ -1620,7 +1620,7 @@ lto_input_mode_table (struct lto_file_decl_data *file_data)
       unsigned int size = bp_unpack_value (&bp, 8);
       unsigned int prec = bp_unpack_value (&bp, 16);
       machine_mode inner = (machine_mode) bp_unpack_value (&bp, 8);
-      unsigned int nunits = bp_unpack_value (&bp, 8);
+      poly_uint16 nunits = bp_unpack_poly_value (&bp, 16);
       unsigned int ibit = 0, fbit = 0;
       unsigned int real_fmt_len = 0;
       const char *real_fmt_name = NULL;
@@ -1658,7 +1658,7 @@ lto_input_mode_table (struct lto_file_decl_data *file_data)
 		  : GET_MODE_INNER (mr) != table[(int) inner])
 	      || GET_MODE_IBIT (mr) != ibit
 	      || GET_MODE_FBIT (mr) != fbit
-	      || GET_MODE_NUNITS (mr) != nunits)
+	      || maybe_ne (GET_MODE_NUNITS (mr), nunits))
 	    continue;
 	  else if ((mclass == MODE_FLOAT || mclass == MODE_DECIMAL_FLOAT)
 		   && strcmp (REAL_MODE_FORMAT (mr)->name, real_fmt_name) != 0)
diff --git a/gcc/lto-streamer-out.c b/gcc/lto-streamer-out.c
index 68305c4..0aea7d5 100644
--- a/gcc/lto-streamer-out.c
+++ b/gcc/lto-streamer-out.c
@@ -2813,7 +2813,7 @@ lto_write_mode_table (void)
 	  bp_pack_value (&bp, GET_MODE_SIZE (m), 8);
 	  bp_pack_value (&bp, GET_MODE_PRECISION (m), 16);
 	  bp_pack_value (&bp, GET_MODE_INNER (m), 8);
-	  bp_pack_value (&bp, GET_MODE_NUNITS (m), 8);
+	  bp_pack_poly_value (&bp, GET_MODE_NUNITS (m), 16);
 	  switch (GET_MODE_CLASS (m))
 	    {
 	    case MODE_FRACT:
diff --git a/gcc/machmode.h b/gcc/machmode.h
index 6d9f542..b97e046 100644
--- a/gcc/machmode.h
+++ b/gcc/machmode.h
@@ -25,7 +25,7 @@ typedef opt_mode<machine_mode> opt_machine_mode;
 extern CONST_MODE_SIZE unsigned short mode_size[NUM_MACHINE_MODES];
 extern const unsigned short mode_precision[NUM_MACHINE_MODES];
 extern const unsigned char mode_inner[NUM_MACHINE_MODES];
-extern const unsigned char mode_nunits[NUM_MACHINE_MODES];
+extern const poly_uint16_pod mode_nunits[NUM_MACHINE_MODES];
 extern CONST_MODE_UNIT_SIZE unsigned char mode_unit_size[NUM_MACHINE_MODES];
 extern const unsigned short mode_unit_precision[NUM_MACHINE_MODES];
 extern const unsigned char mode_wider[NUM_MACHINE_MODES];
@@ -76,6 +76,14 @@ struct mode_traits<machine_mode>
   typedef machine_mode from_int;
 };
 
+/* Always treat machine modes as fixed-size while compiling code specific
+   to targets that have no variable-size modes.  */
+#if defined (IN_TARGET_CODE) && NUM_POLY_INT_COEFFS == 1
+#define ONLY_FIXED_SIZE_MODES 1
+#else
+#define ONLY_FIXED_SIZE_MODES 0
+#endif
+
 /* Get the name of mode MODE as a string.  */
 
 extern const char * const mode_name[NUM_MACHINE_MODES];
@@ -313,6 +321,7 @@ template<typename T>
 struct pod_mode
 {
   typedef typename mode_traits<T>::from_int from_int;
+  typedef typename T::measurement_type measurement_type;
 
   machine_mode m_mode;
   ALWAYS_INLINE operator machine_mode () const { return m_mode; }
@@ -391,6 +400,7 @@ class scalar_int_mode
 {
 public:
   typedef mode_traits<scalar_int_mode>::from_int from_int;
+  typedef unsigned short measurement_type;
 
   ALWAYS_INLINE scalar_int_mode () {}
   ALWAYS_INLINE scalar_int_mode (from_int m) : m_mode (machine_mode (m)) {}
@@ -415,6 +425,7 @@ class scalar_float_mode
 {
 public:
   typedef mode_traits<scalar_float_mode>::from_int from_int;
+  typedef unsigned short measurement_type;
 
   ALWAYS_INLINE scalar_float_mode () {}
   ALWAYS_INLINE scalar_float_mode (from_int m) : m_mode (machine_mode (m)) {}
@@ -439,6 +450,7 @@ class scalar_mode
 {
 public:
   typedef mode_traits<scalar_mode>::from_int from_int;
+  typedef unsigned short measurement_type;
 
   ALWAYS_INLINE scalar_mode () {}
   ALWAYS_INLINE scalar_mode (from_int m) : m_mode (machine_mode (m)) {}
@@ -480,6 +492,7 @@ class complex_mode
 {
 public:
   typedef mode_traits<complex_mode>::from_int from_int;
+  typedef unsigned short measurement_type;
 
   ALWAYS_INLINE complex_mode () {}
   ALWAYS_INLINE complex_mode (from_int m) : m_mode (machine_mode (m)) {}
@@ -570,7 +583,7 @@ mode_to_unit_precision (machine_mode mode)
 
 /* Return the base GET_MODE_NUNITS value for MODE.  */
 
-ALWAYS_INLINE unsigned short
+ALWAYS_INLINE poly_uint16
 mode_to_nunits (machine_mode mode)
 {
 #if GCC_VERSION >= 4001
@@ -627,7 +640,29 @@ extern const unsigned HOST_WIDE_INT mode_mask_array[NUM_MACHINE_MODES];
 /* Get the number of units in an object of mode MODE.  This is 2 for
    complex modes and the number of elements for vector modes.  */
 
-#define GET_MODE_NUNITS(MODE) (mode_to_nunits (MODE))
+#if ONLY_FIXED_SIZE_MODES
+#define GET_MODE_NUNITS(MODE) (mode_to_nunits (MODE).coeffs[0])
+#else
+ALWAYS_INLINE poly_uint16
+GET_MODE_NUNITS (machine_mode mode)
+{
+  return mode_to_nunits (mode);
+}
+
+template<typename T>
+ALWAYS_INLINE typename if_poly<typename T::measurement_type>::type
+GET_MODE_NUNITS (const T &mode)
+{
+  return mode_to_nunits (mode);
+}
+
+template<typename T>
+ALWAYS_INLINE typename if_nonpoly<typename T::measurement_type>::type
+GET_MODE_NUNITS (const T &mode)
+{
+  return mode_to_nunits (mode).coeffs[0];
+}
+#endif
 
 /* Get the next wider natural mode (eg, QI -> HI -> SI -> DI -> TI).  */
 
@@ -660,6 +695,7 @@ class fixed_size_mode
 {
 public:
   typedef mode_traits<fixed_size_mode>::from_int from_int;
+  typedef unsigned short measurement_type;
 
   ALWAYS_INLINE fixed_size_mode () {}
   ALWAYS_INLINE fixed_size_mode (from_int m) : m_mode (machine_mode (m)) {}
diff --git a/gcc/optabs-query.c b/gcc/optabs-query.c
index 63139ff..cae9c30 100644
--- a/gcc/optabs-query.c
+++ b/gcc/optabs-query.c
@@ -391,7 +391,7 @@ can_vec_perm_var_p (machine_mode mode)
   /* We allow fallback to a QI vector mode, and adjust the mask.  */
   machine_mode qimode;
   if (!qimode_for_vec_perm (mode).exists (&qimode)
-      || GET_MODE_NUNITS (qimode) > GET_MODE_MASK (QImode) + 1)
+      || maybe_gt (GET_MODE_NUNITS (qimode), GET_MODE_MASK (QImode) + 1))
     return false;
 
   if (direct_optab_handler (vec_perm_optab, qimode) == CODE_FOR_nothing)
@@ -498,7 +498,6 @@ int
 can_mult_highpart_p (machine_mode mode, bool uns_p)
 {
   optab op;
-  unsigned i, nunits;
 
   op = uns_p ? umul_highpart_optab : smul_highpart_optab;
   if (optab_handler (op, mode) != CODE_FOR_nothing)
@@ -508,7 +507,7 @@ can_mult_highpart_p (machine_mode mode, bool uns_p)
   if (GET_MODE_CLASS (mode) != MODE_VECTOR_INT)
     return 0;
 
-  nunits = GET_MODE_NUNITS (mode);
+  poly_int64 nunits = GET_MODE_NUNITS (mode);
 
   op = uns_p ? vec_widen_umult_even_optab : vec_widen_smult_even_optab;
   if (optab_handler (op, mode) != CODE_FOR_nothing)
@@ -518,7 +517,7 @@ can_mult_highpart_p (machine_mode mode, bool uns_p)
 	{
 	  /* The encoding has 2 interleaved stepped patterns.  */
 	  vec_perm_builder sel (nunits, 2, 3);
-	  for (i = 0; i < 6; ++i)
+	  for (unsigned int i = 0; i < 6; ++i)
 	    sel.quick_push (!BYTES_BIG_ENDIAN
 			    + (i & ~1)
 			    + ((i & 1) ? nunits : 0));
@@ -536,7 +535,7 @@ can_mult_highpart_p (machine_mode mode, bool uns_p)
 	{
 	  /* The encoding has a single stepped pattern.  */
 	  vec_perm_builder sel (nunits, 1, 3);
-	  for (int i = 0; i < 3; ++i)
+	  for (unsigned int i = 0; i < 3; ++i)
 	    sel.quick_push (2 * i + (BYTES_BIG_ENDIAN ? 0 : 1));
 	  vec_perm_indices indices (sel, 2, nunits);
 	  if (can_vec_perm_const_p (mode, indices))
diff --git a/gcc/optabs-tree.c b/gcc/optabs-tree.c
index 323acde0..69448cf 100644
--- a/gcc/optabs-tree.c
+++ b/gcc/optabs-tree.c
@@ -328,7 +328,7 @@ expand_vec_cond_expr_p (tree value_type, tree cmp_op_type, enum tree_code code)
     return true;
 
   if (GET_MODE_SIZE (value_mode) != GET_MODE_SIZE (cmp_op_mode)
-      || GET_MODE_NUNITS (value_mode) != GET_MODE_NUNITS (cmp_op_mode))
+      || maybe_ne (GET_MODE_NUNITS (value_mode), GET_MODE_NUNITS (cmp_op_mode)))
     return false;
 
   if (get_vcond_icode (TYPE_MODE (value_type), TYPE_MODE (cmp_op_type),
diff --git a/gcc/optabs.c b/gcc/optabs.c
index 7a3cf5c..5a85d78 100644
--- a/gcc/optabs.c
+++ b/gcc/optabs.c
@@ -371,17 +371,15 @@ force_expand_binop (machine_mode mode, optab binoptab,
 rtx
 expand_vector_broadcast (machine_mode vmode, rtx op)
 {
-  enum insn_code icode;
+  int n;
   rtvec vec;
-  rtx ret;
-  int i, n;
 
   gcc_checking_assert (VECTOR_MODE_P (vmode));
 
   if (valid_for_const_vector_p (vmode, op))
     return gen_const_vec_duplicate (vmode, op);
 
-  icode = optab_handler (vec_duplicate_optab, vmode);
+  insn_code icode = optab_handler (vec_duplicate_optab, vmode);
   if (icode != CODE_FOR_nothing)
     {
       struct expand_operand ops[2];
@@ -391,6 +389,9 @@ expand_vector_broadcast (machine_mode vmode, rtx op)
       return ops[0].value;
     }
 
+  if (!GET_MODE_NUNITS (vmode).is_constant (&n))
+    return NULL;
+
   /* ??? If the target doesn't have a vec_init, then we have no easy way
      of performing this operation.  Most of this sort of generic support
      is hidden away in the vector lowering support in gimple.  */
@@ -399,11 +400,10 @@ expand_vector_broadcast (machine_mode vmode, rtx op)
   if (icode == CODE_FOR_nothing)
     return NULL;
 
-  n = GET_MODE_NUNITS (vmode);
   vec = rtvec_alloc (n);
-  for (i = 0; i < n; ++i)
+  for (int i = 0; i < n; ++i)
     RTVEC_ELT (vec, i) = op;
-  ret = gen_reg_rtx (vmode);
+  rtx ret = gen_reg_rtx (vmode);
   emit_insn (GEN_FCN (icode) (ret, gen_rtx_PARALLEL (vmode, vec)));
 
   return ret;
@@ -1074,7 +1074,7 @@ expand_binop_directly (enum insn_code icode, machine_mode mode, optab binoptab,
 	 arguments.  */
       tmp_mode = insn_data[(int) icode].operand[0].mode;
       if (VECTOR_MODE_P (mode)
-	  && GET_MODE_NUNITS (tmp_mode) != 2 * GET_MODE_NUNITS (mode))
+	  && maybe_ne (GET_MODE_NUNITS (tmp_mode), 2 * GET_MODE_NUNITS (mode)))
 	{
 	  delete_insns_since (last);
 	  return NULL_RTX;
@@ -5396,22 +5396,26 @@ vector_compare_rtx (machine_mode cmp_mode, enum tree_code tcode,
 static rtx
 shift_amt_for_vec_perm_mask (machine_mode mode, const vec_perm_indices &sel)
 {
-  unsigned int nelt = GET_MODE_NUNITS (mode);
   unsigned int bitsize = GET_MODE_UNIT_BITSIZE (mode);
   poly_int64 first = sel[0];
-  if (maybe_ge (sel[0], nelt))
+  if (maybe_ge (sel[0], GET_MODE_NUNITS (mode)))
     return NULL_RTX;
 
   if (!sel.series_p (0, 1, first, 1))
-    for (unsigned int i = 1; i < nelt; i++)
-      {
-	poly_int64 expected = i + first;
-	/* Indices into the second vector are all equivalent.  */
-	if (maybe_lt (sel[i], nelt)
-	    ? maybe_ne (sel[i], expected)
-	    : maybe_lt (expected, nelt))
-	  return NULL_RTX;
-      }
+    {
+      unsigned int nelt;
+      if (!GET_MODE_NUNITS (mode).is_constant (&nelt))
+	return NULL_RTX;
+      for (unsigned int i = 1; i < nelt; i++)
+	{
+	  poly_int64 expected = i + first;
+	  /* Indices into the second vector are all equivalent.  */
+	  if (maybe_lt (sel[i], nelt)
+	      ? maybe_ne (sel[i], expected)
+	      : maybe_lt (expected, nelt))
+	    return NULL_RTX;
+	}
+    }
 
   return gen_int_shift_amount (mode, first * bitsize);
 }
@@ -5631,7 +5635,7 @@ expand_vec_perm_var (machine_mode mode, rtx v0, rtx v1, rtx sel, rtx target)
      permutation to a byte-based permutation and try again.  */
   machine_mode qimode;
   if (!qimode_for_vec_perm (mode).exists (&qimode)
-      || GET_MODE_NUNITS (qimode) > GET_MODE_MASK (QImode) + 1)
+      || maybe_gt (GET_MODE_NUNITS (qimode), GET_MODE_MASK (QImode) + 1))
     return NULL_RTX;
   icode = direct_optab_handler (vec_perm_optab, qimode);
   if (icode == CODE_FOR_nothing)
@@ -5755,7 +5759,8 @@ expand_vec_cond_expr (tree vec_cond_type, tree op0, tree op1, tree op2,
 
 
   gcc_assert (GET_MODE_SIZE (mode) == GET_MODE_SIZE (cmp_op_mode)
-	      && GET_MODE_NUNITS (mode) == GET_MODE_NUNITS (cmp_op_mode));
+	      && known_eq (GET_MODE_NUNITS (mode),
+			   GET_MODE_NUNITS (cmp_op_mode)));
 
   icode = get_vcond_icode (mode, cmp_op_mode, unsignedp);
   if (icode == CODE_FOR_nothing)
@@ -5850,7 +5855,7 @@ expand_mult_highpart (machine_mode mode, rtx op0, rtx op1,
 {
   struct expand_operand eops[3];
   enum insn_code icode;
-  int method, i, nunits;
+  int method, i;
   machine_mode wmode;
   rtx m1, m2;
   optab tab1, tab2;
@@ -5879,9 +5884,9 @@ expand_mult_highpart (machine_mode mode, rtx op0, rtx op1,
     }
 
   icode = optab_handler (tab1, mode);
-  nunits = GET_MODE_NUNITS (mode);
   wmode = insn_data[icode].operand[0].mode;
-  gcc_checking_assert (2 * GET_MODE_NUNITS (wmode) == nunits);
+  gcc_checking_assert (known_eq (2 * GET_MODE_NUNITS (wmode),
+				 GET_MODE_NUNITS (mode)));
   gcc_checking_assert (GET_MODE_SIZE (wmode) == GET_MODE_SIZE (mode));
 
   create_output_operand (&eops[0], gen_reg_rtx (wmode), wmode);
@@ -5900,15 +5905,15 @@ expand_mult_highpart (machine_mode mode, rtx op0, rtx op1,
   if (method == 2)
     {
       /* The encoding has 2 interleaved stepped patterns.  */
-      sel.new_vector (nunits, 2, 3);
+      sel.new_vector (GET_MODE_NUNITS (mode), 2, 3);
       for (i = 0; i < 6; ++i)
 	sel.quick_push (!BYTES_BIG_ENDIAN + (i & ~1)
-			+ ((i & 1) ? nunits : 0));
+			+ ((i & 1) ? GET_MODE_NUNITS (mode) : 0));
     }
   else
     {
       /* The encoding has a single interleaved stepped pattern.  */
-      sel.new_vector (nunits, 1, 3);
+      sel.new_vector (GET_MODE_NUNITS (mode), 1, 3);
       for (i = 0; i < 3; ++i)
 	sel.quick_push (2 * i + (BYTES_BIG_ENDIAN ? 0 : 1));
     }
diff --git a/gcc/rtlanal.c b/gcc/rtlanal.c
index 1dad066..33a0980 100644
--- a/gcc/rtlanal.c
+++ b/gcc/rtlanal.c
@@ -3706,11 +3706,11 @@ subreg_get_info (unsigned int xregno, machine_mode xmode,
   if (HARD_REGNO_NREGS_HAS_PADDING (xregno, xmode))
     {
       /* As a consequence, we must be dealing with a constant number of
-	 scalars, and thus a constant offset.  */
+	 scalars, and thus a constant offset and number of units.  */
       HOST_WIDE_INT coffset = offset.to_constant ();
       HOST_WIDE_INT cysize = ysize.to_constant ();
       nregs_xmode = HARD_REGNO_NREGS_WITH_PADDING (xregno, xmode);
-      unsigned int nunits = GET_MODE_NUNITS (xmode);
+      unsigned int nunits = GET_MODE_NUNITS (xmode).to_constant ();
       scalar_mode xmode_unit = GET_MODE_INNER (xmode);
       gcc_assert (HARD_REGNO_NREGS_HAS_PADDING (xregno, xmode_unit));
       gcc_assert (nregs_xmode
diff --git a/gcc/rtx-vector-builder.c b/gcc/rtx-vector-builder.c
index ed75263..33f76d0 100644
--- a/gcc/rtx-vector-builder.c
+++ b/gcc/rtx-vector-builder.c
@@ -65,7 +65,9 @@ rtx_vector_builder::build ()
   if (x)
     return x;
 
-  unsigned int nelts = GET_MODE_NUNITS (m_mode);
+  unsigned int nelts;
+  if (!GET_MODE_NUNITS (m_mode).is_constant (&nelts))
+    nelts = encoded_nelts ();
   rtvec v = rtvec_alloc (nelts);
   for (unsigned int i = 0; i < nelts; ++i)
     RTVEC_ELT (v, i) = elt (i);
diff --git a/gcc/simplify-rtx.c b/gcc/simplify-rtx.c
index 7cfe8f39..a20782a 100644
--- a/gcc/simplify-rtx.c
+++ b/gcc/simplify-rtx.c
@@ -1223,7 +1223,7 @@ simplify_unary_operation_1 (enum rtx_code code, machine_mode mode, rtx op)
 
       /* If we know that the value is already truncated, we can
 	 replace the TRUNCATE with a SUBREG.  */
-      if (GET_MODE_NUNITS (mode) == 1
+      if (known_eq (GET_MODE_NUNITS (mode), 1)
 	  && (TRULY_NOOP_TRUNCATION_MODES_P (mode, GET_MODE (op))
 	      || truncated_to_mode (mode, op)))
 	{
@@ -1739,10 +1739,13 @@ simplify_const_unary_operation (enum rtx_code code, machine_mode mode,
       }
       if (CONST_SCALAR_INT_P (op) || CONST_DOUBLE_AS_FLOAT_P (op))
 	return gen_const_vec_duplicate (mode, op);
-      if (GET_CODE (op) == CONST_VECTOR)
+      unsigned int n_elts;
+      if (GET_CODE (op) == CONST_VECTOR
+	  && GET_MODE_NUNITS (mode).is_constant (&n_elts))
 	{
-	  unsigned int n_elts = GET_MODE_NUNITS (mode);
-	  unsigned int in_n_elts = CONST_VECTOR_NUNITS (op);
+	  /* This must be constant if we're duplicating it to a constant
+	     number of elements.  */
+	  unsigned int in_n_elts = CONST_VECTOR_NUNITS (op).to_constant ();
 	  gcc_assert (in_n_elts < n_elts);
 	  gcc_assert ((n_elts % in_n_elts) == 0);
 	  rtvec v = rtvec_alloc (n_elts);
@@ -1754,15 +1757,17 @@ simplify_const_unary_operation (enum rtx_code code, machine_mode mode,
 
   if (VECTOR_MODE_P (mode) && GET_CODE (op) == CONST_VECTOR)
     {
-      int elt_size = GET_MODE_UNIT_SIZE (mode);
-      unsigned n_elts = (GET_MODE_SIZE (mode) / elt_size);
+      unsigned int n_elts;
+      if (!CONST_VECTOR_NUNITS (op).is_constant (&n_elts))
+	return NULL_RTX;
+
       machine_mode opmode = GET_MODE (op);
-      int op_elt_size = GET_MODE_UNIT_SIZE (opmode);
-      unsigned op_n_elts = (GET_MODE_SIZE (opmode) / op_elt_size);
+      gcc_assert (known_eq (GET_MODE_NUNITS (mode), n_elts));
+      gcc_assert (known_eq (GET_MODE_NUNITS (opmode), n_elts));
+
       rtvec v = rtvec_alloc (n_elts);
       unsigned int i;
 
-      gcc_assert (op_n_elts == n_elts);
       for (i = 0; i < n_elts; i++)
 	{
 	  rtx x = simplify_unary_operation (code, GET_MODE_INNER (mode),
@@ -3617,13 +3622,14 @@ simplify_binary_operation_1 (enum rtx_code code, machine_mode mode,
 	     nested VEC_SELECT expressions.  When input operand is a memory
 	     operand, this operation can be simplified to a simple scalar
 	     load from an offseted memory address.  */
-	  if (GET_CODE (trueop0) == VEC_SELECT)
+	  int n_elts;
+	  if (GET_CODE (trueop0) == VEC_SELECT
+	      && (GET_MODE_NUNITS (GET_MODE (XEXP (trueop0, 0)))
+		  .is_constant (&n_elts)))
 	    {
 	      rtx op0 = XEXP (trueop0, 0);
 	      rtx op1 = XEXP (trueop0, 1);
 
-	      int n_elts = GET_MODE_NUNITS (GET_MODE (op0));
-
 	      int i = INTVAL (XVECEXP (trueop1, 0, 0));
 	      int elem;
 
@@ -3648,9 +3654,11 @@ simplify_binary_operation_1 (enum rtx_code code, machine_mode mode,
 		  mode00 = GET_MODE (op00);
 		  mode01 = GET_MODE (op01);
 
-		  /* Find out number of elements of each operand.  */
-		  n_elts00 = GET_MODE_NUNITS (mode00);
-		  n_elts01 = GET_MODE_NUNITS (mode01);
+		  /* Find out the number of elements of each operand.
+		     Since the concatenated result has a constant number
+		     of elements, the operands must too.  */
+		  n_elts00 = GET_MODE_NUNITS (mode00).to_constant ();
+		  n_elts01 = GET_MODE_NUNITS (mode01).to_constant ();
 
 		  gcc_assert (n_elts == n_elts00 + n_elts01);
 
@@ -3689,12 +3697,11 @@ simplify_binary_operation_1 (enum rtx_code code, machine_mode mode,
 
 	  if (GET_CODE (trueop0) == CONST_VECTOR)
 	    {
-	      int elt_size = GET_MODE_UNIT_SIZE (mode);
-	      unsigned n_elts = (GET_MODE_SIZE (mode) / elt_size);
+	      unsigned n_elts = XVECLEN (trueop1, 0);
 	      rtvec v = rtvec_alloc (n_elts);
 	      unsigned int i;
 
-	      gcc_assert (XVECLEN (trueop1, 0) == (int) n_elts);
+	      gcc_assert (known_eq (n_elts, GET_MODE_NUNITS (mode)));
 	      for (i = 0; i < n_elts; i++)
 		{
 		  rtx x = XVECEXP (trueop1, 0, i);
@@ -3763,15 +3770,18 @@ simplify_binary_operation_1 (enum rtx_code code, machine_mode mode,
 	    }
 
 	  /* If we select one half of a vec_concat, return that.  */
+	  int l0, l1;
 	  if (GET_CODE (trueop0) == VEC_CONCAT
+	      && (GET_MODE_NUNITS (GET_MODE (XEXP (trueop0, 0)))
+		  .is_constant (&l0))
+	      && (GET_MODE_NUNITS (GET_MODE (XEXP (trueop0, 1)))
+		  .is_constant (&l1))
 	      && CONST_INT_P (XVECEXP (trueop1, 0, 0)))
 	    {
 	      rtx subop0 = XEXP (trueop0, 0);
 	      rtx subop1 = XEXP (trueop0, 1);
 	      machine_mode mode0 = GET_MODE (subop0);
 	      machine_mode mode1 = GET_MODE (subop1);
-	      int l0 = GET_MODE_NUNITS (mode0);
-	      int l1 = GET_MODE_NUNITS (mode1);
 	      int i0 = INTVAL (XVECEXP (trueop1, 0, 0));
 	      if (i0 == 0 && !side_effects_p (op1) && mode == mode0)
 		{
@@ -3878,7 +3888,7 @@ simplify_binary_operation_1 (enum rtx_code code, machine_mode mode,
 	{
 	  rtx op0_subop1 = XEXP (trueop0, 1);
 	  gcc_assert (GET_CODE (op0_subop1) == PARALLEL);
-	  gcc_assert (XVECLEN (trueop1, 0) == GET_MODE_NUNITS (mode));
+	  gcc_assert (known_eq (XVECLEN (trueop1, 0), GET_MODE_NUNITS (mode)));
 
 	  /* Apply the outer ordering vector to the inner one.  (The inner
 	     ordering vector is expressly permitted to be of a different
@@ -3922,15 +3932,16 @@ simplify_binary_operation_1 (enum rtx_code code, machine_mode mode,
 	else
 	  gcc_assert (GET_MODE_INNER (mode) == op1_mode);
 
+	unsigned int n_elts, in_n_elts;
 	if ((GET_CODE (trueop0) == CONST_VECTOR
 	     || CONST_SCALAR_INT_P (trueop0) 
 	     || CONST_DOUBLE_AS_FLOAT_P (trueop0))
 	    && (GET_CODE (trueop1) == CONST_VECTOR
 		|| CONST_SCALAR_INT_P (trueop1) 
-		|| CONST_DOUBLE_AS_FLOAT_P (trueop1)))
+		|| CONST_DOUBLE_AS_FLOAT_P (trueop1))
+	    && GET_MODE_NUNITS (mode).is_constant (&n_elts)
+	    && GET_MODE_NUNITS (op0_mode).is_constant (&in_n_elts))
 	  {
-	    unsigned n_elts = GET_MODE_NUNITS (mode);
-	    unsigned in_n_elts = GET_MODE_NUNITS (op0_mode);
 	    rtvec v = rtvec_alloc (n_elts);
 	    unsigned int i;
 	    for (i = 0; i < n_elts; i++)
@@ -4020,9 +4031,12 @@ simplify_const_binary_operation (enum rtx_code code, machine_mode mode,
       && GET_CODE (op0) == CONST_VECTOR
       && GET_CODE (op1) == CONST_VECTOR)
     {
-      unsigned int n_elts = CONST_VECTOR_NUNITS (op0);
-      gcc_assert (n_elts == (unsigned int) CONST_VECTOR_NUNITS (op1));
-      gcc_assert (n_elts == GET_MODE_NUNITS (mode));
+      unsigned int n_elts;
+      if (!CONST_VECTOR_NUNITS (op0).is_constant (&n_elts))
+	return NULL_RTX;
+
+      gcc_assert (known_eq (n_elts, CONST_VECTOR_NUNITS (op1)));
+      gcc_assert (known_eq (n_elts, GET_MODE_NUNITS (mode)));
       rtvec v = rtvec_alloc (n_elts);
       unsigned int i;
 
@@ -4048,7 +4062,9 @@ simplify_const_binary_operation (enum rtx_code code, machine_mode mode,
 	  || CONST_DOUBLE_AS_FLOAT_P (op1)
 	  || CONST_FIXED_P (op1)))
     {
-      unsigned n_elts = GET_MODE_NUNITS (mode);
+      /* Both inputs have a constant number of elements, so the result
+	 must too.  */
+      unsigned n_elts = GET_MODE_NUNITS (mode).to_constant ();
       rtvec v = rtvec_alloc (n_elts);
 
       gcc_assert (n_elts >= 2);
@@ -4062,8 +4078,8 @@ simplify_const_binary_operation (enum rtx_code code, machine_mode mode,
 	}
       else
 	{
-	  unsigned op0_n_elts = GET_MODE_NUNITS (GET_MODE (op0));
-	  unsigned op1_n_elts = GET_MODE_NUNITS (GET_MODE (op1));
+	  unsigned op0_n_elts = GET_MODE_NUNITS (GET_MODE (op0)).to_constant ();
+	  unsigned op1_n_elts = GET_MODE_NUNITS (GET_MODE (op1)).to_constant ();
 	  unsigned i;
 
 	  gcc_assert (GET_CODE (op0) == CONST_VECTOR);
@@ -5566,6 +5582,7 @@ simplify_ternary_operation (enum rtx_code code, machine_mode mode,
   bool any_change = false;
   rtx tem, trueop2;
   scalar_int_mode int_mode, int_op0_mode;
+  unsigned int n_elts;
 
   switch (code)
     {
@@ -5668,14 +5685,17 @@ simplify_ternary_operation (enum rtx_code code, machine_mode mode,
 	  && GET_CODE (XEXP (op0, 1)) == CONST_VECTOR)
 	{
 	  rtx cv = XEXP (op0, 1);
-	  int nunits = CONST_VECTOR_NUNITS (cv);
+	  int nunits;
 	  bool ok = true;
-	  for (int i = 0; i < nunits; ++i)
-	    if (CONST_VECTOR_ELT (cv, i) != const0_rtx)
-	      {
-		ok = false;
-		break;
-	      }
+	  if (!CONST_VECTOR_NUNITS (cv).is_constant (&nunits))
+	    ok = false;
+	  else
+	    for (int i = 0; i < nunits; ++i)
+	      if (CONST_VECTOR_ELT (cv, i) != const0_rtx)
+		{
+		  ok = false;
+		  break;
+		}
 	  if (ok)
 	    {
 	      rtx new_op0 = gen_rtx_NE (GET_MODE (op0),
@@ -5749,9 +5769,9 @@ simplify_ternary_operation (enum rtx_code code, machine_mode mode,
       gcc_assert (GET_MODE (op1) == mode);
       gcc_assert (VECTOR_MODE_P (mode));
       trueop2 = avoid_constant_pool_reference (op2);
-      if (CONST_INT_P (trueop2))
+      if (CONST_INT_P (trueop2)
+	  && GET_MODE_NUNITS (mode).is_constant (&n_elts))
 	{
-	  unsigned n_elts = GET_MODE_NUNITS (mode);
 	  unsigned HOST_WIDE_INT sel = UINTVAL (trueop2);
 	  unsigned HOST_WIDE_INT mask;
 	  if (n_elts == HOST_BITS_PER_WIDE_INT)
@@ -5815,7 +5835,7 @@ simplify_ternary_operation (enum rtx_code code, machine_mode mode,
 	  if (GET_CODE (op0) == VEC_DUPLICATE
 	      && GET_CODE (XEXP (op0, 0)) == VEC_SELECT
 	      && GET_CODE (XEXP (XEXP (op0, 0), 1)) == PARALLEL
-	      && mode_nunits[GET_MODE (XEXP (op0, 0))] == 1)
+	      && known_eq (GET_MODE_NUNITS (GET_MODE (XEXP (op0, 0))), 1))
 	    {
 	      tem = XVECEXP ((XEXP (XEXP (op0, 0), 1)), 0, 0);
 	      if (CONST_INT_P (tem) && CONST_INT_P (op2))
@@ -5831,8 +5851,8 @@ simplify_ternary_operation (enum rtx_code code, machine_mode mode,
 	     (vec_concat (A) (X)) if N == 2.  */
 	  if (GET_CODE (op0) == VEC_DUPLICATE
 	      && GET_CODE (op1) == CONST_VECTOR
-	      && CONST_VECTOR_NUNITS (op1) == 2
-	      && GET_MODE_NUNITS (GET_MODE (op0)) == 2
+	      && known_eq (CONST_VECTOR_NUNITS (op1), 2)
+	      && known_eq (GET_MODE_NUNITS (GET_MODE (op0)), 2)
 	      && IN_RANGE (sel, 1, 2))
 	    {
 	      rtx newop0 = XEXP (op0, 0);
@@ -5846,8 +5866,8 @@ simplify_ternary_operation (enum rtx_code code, machine_mode mode,
 	     Only applies for vectors of two elements.  */
 	  if (GET_CODE (op0) == VEC_DUPLICATE
 	      && GET_CODE (op1) == VEC_CONCAT
-	      && GET_MODE_NUNITS (GET_MODE (op0)) == 2
-	      && GET_MODE_NUNITS (GET_MODE (op1)) == 2
+	      && known_eq (GET_MODE_NUNITS (GET_MODE (op0)), 2)
+	      && known_eq (GET_MODE_NUNITS (GET_MODE (op1)), 2)
 	      && IN_RANGE (sel, 1, 2))
 	    {
 	      rtx newop0 = XEXP (op0, 0);
@@ -5881,8 +5901,8 @@ simplify_ternary_operation (enum rtx_code code, machine_mode mode,
 	  if ((GET_CODE (op0) == VEC_DUPLICATE
 	       || GET_CODE (op1) == VEC_DUPLICATE)
 	      && GET_MODE (op0) == GET_MODE (op1)
-	      && GET_MODE_NUNITS (GET_MODE (op0)) == 2
-	      && GET_MODE_NUNITS (GET_MODE (op1)) == 2
+	      && known_eq (GET_MODE_NUNITS (GET_MODE (op0)), 2)
+	      && known_eq (GET_MODE_NUNITS (GET_MODE (op1)), 2)
 	      && IN_RANGE (sel, 1, 2))
 	    {
 	      rtx newop0 = op0, newop1 = op1;
@@ -5918,8 +5938,8 @@ simplify_ternary_operation (enum rtx_code code, machine_mode mode,
 	     of N.  */
 	  if (GET_CODE (op0) == VEC_DUPLICATE
 	      && GET_CODE (op1) == VEC_DUPLICATE
-	      && GET_MODE_NUNITS (GET_MODE (op0)) == 2
-	      && GET_MODE_NUNITS (GET_MODE (op1)) == 2
+	      && known_eq (GET_MODE_NUNITS (GET_MODE (op0)), 2)
+	      && known_eq (GET_MODE_NUNITS (GET_MODE (op1)), 2)
 	      && IN_RANGE (sel, 1, 2))
 	    {
 	      rtx newop0 = XEXP (op0, 0);
@@ -5990,7 +6010,7 @@ simplify_immed_subreg (fixed_size_mode outermode, rtx op,
 
   if (GET_CODE (op) == CONST_VECTOR)
     {
-      num_elem = CONST_VECTOR_NUNITS (op);
+      num_elem = GET_MODE_NUNITS (innermode);
       elem_bitsize = GET_MODE_UNIT_BITSIZE (innermode);
     }
   else
@@ -6679,7 +6699,7 @@ test_vector_ops_duplicate (machine_mode mode, rtx scalar_reg)
 {
   scalar_mode inner_mode = GET_MODE_INNER (mode);
   rtx duplicate = gen_rtx_VEC_DUPLICATE (mode, scalar_reg);
-  unsigned int nunits = GET_MODE_NUNITS (mode);
+  poly_uint64 nunits = GET_MODE_NUNITS (mode);
   if (GET_MODE_CLASS (mode) == MODE_VECTOR_INT)
     {
       /* Test some simple unary cases with VEC_DUPLICATE arguments.  */
@@ -6716,11 +6736,15 @@ test_vector_ops_duplicate (machine_mode mode, rtx scalar_reg)
 						duplicate, zero_par));
 
   /* And again with the final element.  */
-  rtx last_index = gen_int_mode (GET_MODE_NUNITS (mode) - 1, word_mode);
-  rtx last_par = gen_rtx_PARALLEL (VOIDmode, gen_rtvec (1, last_index));
-  ASSERT_RTX_PTR_EQ (scalar_reg,
-		     simplify_binary_operation (VEC_SELECT, inner_mode,
-						duplicate, last_par));
+  unsigned HOST_WIDE_INT const_nunits;
+  if (nunits.is_constant (&const_nunits))
+    {
+      rtx last_index = gen_int_mode (const_nunits - 1, word_mode);
+      rtx last_par = gen_rtx_PARALLEL (VOIDmode, gen_rtvec (1, last_index));
+      ASSERT_RTX_PTR_EQ (scalar_reg,
+			 simplify_binary_operation (VEC_SELECT, inner_mode,
+						    duplicate, last_par));
+    }
 
   /* Test a scalar subreg of a VEC_DUPLICATE.  */
   poly_uint64 offset = subreg_lowpart_offset (inner_mode, mode);
@@ -6729,7 +6753,8 @@ test_vector_ops_duplicate (machine_mode mode, rtx scalar_reg)
 				      mode, offset));
 
   machine_mode narrower_mode;
-  if (nunits > 2
+  if (maybe_ne (nunits, 2U)
+      && multiple_p (nunits, 2)
       && mode_for_vector (inner_mode, 2).exists (&narrower_mode)
       && VECTOR_MODE_P (narrower_mode))
     {
@@ -6820,7 +6845,7 @@ test_vector_ops ()
 	  rtx scalar_reg = make_test_reg (GET_MODE_INNER (mode));
 	  test_vector_ops_duplicate (mode, scalar_reg);
 	  if (GET_MODE_CLASS (mode) == MODE_VECTOR_INT
-	      && GET_MODE_NUNITS (mode) > 2)
+	      && maybe_gt (GET_MODE_NUNITS (mode), 2))
 	    test_vector_ops_series (mode, scalar_reg);
 	}
     }
diff --git a/gcc/tree-vect-data-refs.c b/gcc/tree-vect-data-refs.c
index bcec96f..4de0864 100644
--- a/gcc/tree-vect-data-refs.c
+++ b/gcc/tree-vect-data-refs.c
@@ -4582,12 +4582,22 @@ vect_grouped_store_supported (tree vectype, unsigned HOST_WIDE_INT count)
   /* Check that the permutation is supported.  */
   if (VECTOR_MODE_P (mode))
     {
-      unsigned int i, nelt = GET_MODE_NUNITS (mode);
+      unsigned int i;
       if (count == 3)
 	{
 	  unsigned int j0 = 0, j1 = 0, j2 = 0;
 	  unsigned int i, j;
 
+	  unsigned int nelt;
+	  if (!GET_MODE_NUNITS (mode).is_constant (&nelt))
+	    {
+	      if (dump_enabled_p ())
+		dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location,
+				 "cannot handle groups of 3 stores for"
+				 " variable-length vectors\n");
+	      return false;
+	    }
+
 	  vec_perm_builder sel (nelt, nelt, 1);
 	  sel.quick_grow (nelt);
 	  vec_perm_indices indices;
@@ -4638,6 +4648,7 @@ vect_grouped_store_supported (tree vectype, unsigned HOST_WIDE_INT count)
 	{
 	  /* If length is not equal to 3 then only power of 2 is supported.  */
 	  gcc_assert (pow2p_hwi (count));
+	  poly_uint64 nelt = GET_MODE_NUNITS (mode);
 
 	  /* The encoding has 2 interleaved stepped patterns.  */
 	  vec_perm_builder sel (nelt, 2, 3);
@@ -4651,7 +4662,7 @@ vect_grouped_store_supported (tree vectype, unsigned HOST_WIDE_INT count)
 	  if (can_vec_perm_const_p (mode, indices))
 	    {
 	      for (i = 0; i < 6; i++)
-		sel[i] += nelt / 2;
+		sel[i] += exact_div (nelt, 2);
 	      indices.new_vector (sel, 2, nelt);
 	      if (can_vec_perm_const_p (mode, indices))
 		return true;
@@ -5184,10 +5195,19 @@ vect_grouped_load_supported (tree vectype, bool single_element_p,
   /* Check that the permutation is supported.  */
   if (VECTOR_MODE_P (mode))
     {
-      unsigned int i, j, nelt = GET_MODE_NUNITS (mode);
-
+      unsigned int i, j;
       if (count == 3)
 	{
+	  unsigned int nelt;
+	  if (!GET_MODE_NUNITS (mode).is_constant (&nelt))
+	    {
+	      if (dump_enabled_p ())
+		dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location,
+				 "cannot handle groups of 3 loads for"
+				 " variable-length vectors\n");
+	      return false;
+	    }
+
 	  vec_perm_builder sel (nelt, nelt, 1);
 	  sel.quick_grow (nelt);
 	  vec_perm_indices indices;
@@ -5229,6 +5249,7 @@ vect_grouped_load_supported (tree vectype, bool single_element_p,
 	{
 	  /* If length is not equal to 3 then only power of 2 is supported.  */
 	  gcc_assert (pow2p_hwi (count));
+	  poly_uint64 nelt = GET_MODE_NUNITS (mode);
 
 	  /* The encoding has a single stepped pattern.  */
 	  vec_perm_builder sel (nelt, 1, 3);
diff --git a/gcc/tree-vect-generic.c b/gcc/tree-vect-generic.c
index a7712f6..c0cc656 100644
--- a/gcc/tree-vect-generic.c
+++ b/gcc/tree-vect-generic.c
@@ -1160,7 +1160,7 @@ type_for_widest_vector_mode (tree type, optab op)
 {
   machine_mode inner_mode = TYPE_MODE (type);
   machine_mode best_mode = VOIDmode, mode;
-  int best_nunits = 0;
+  poly_int64 best_nunits = 0;
 
   if (SCALAR_FLOAT_MODE_P (inner_mode))
     mode = MIN_MODE_VECTOR_FLOAT;
@@ -1177,7 +1177,7 @@ type_for_widest_vector_mode (tree type, optab op)
 
   FOR_EACH_MODE_FROM (mode, mode)
     if (GET_MODE_INNER (mode) == inner_mode
-        && GET_MODE_NUNITS (mode) > best_nunits
+	&& maybe_gt (GET_MODE_NUNITS (mode), best_nunits)
 	&& optab_handler (op, mode) != CODE_FOR_nothing)
       best_mode = mode, best_nunits = GET_MODE_NUNITS (mode);
 
diff --git a/gcc/tree-vect-loop.c b/gcc/tree-vect-loop.c
index 82890ab..1f15a64 100644
--- a/gcc/tree-vect-loop.c
+++ b/gcc/tree-vect-loop.c
@@ -3760,10 +3760,14 @@ have_whole_vector_shift (machine_mode mode)
   if (optab_handler (vec_shr_optab, mode) != CODE_FOR_nothing)
     return true;
 
-  unsigned int i, nelt = GET_MODE_NUNITS (mode);
+  /* Variable-length vectors should be handled via the optab.  */
+  unsigned int nelt;
+  if (!GET_MODE_NUNITS (mode).is_constant (&nelt))
+    return false;
+
   vec_perm_builder sel;
   vec_perm_indices indices;
-  for (i = nelt/2; i >= 1; i/=2)
+  for (unsigned int i = nelt / 2; i >= 1; i /= 2)
     {
       calc_vec_perm_mask_for_shift (i, nelt, &sel);
       indices.new_vector (sel, 2, nelt);
diff --git a/gcc/tree.c b/gcc/tree.c
index b75fec3..c9ca760 100644
--- a/gcc/tree.c
+++ b/gcc/tree.c
@@ -9598,19 +9598,19 @@ omp_clause_operand_check_failed (int idx, const_tree t, const char *file,
 }
 #endif /* ENABLE_TREE_CHECKING */
 
-/* Create a new vector type node holding SUBPARTS units of type INNERTYPE,
+/* Create a new vector type node holding NUNITS units of type INNERTYPE,
    and mapped to the machine mode MODE.  Initialize its fields and build
    the information necessary for debugging output.  */
 
 static tree
-make_vector_type (tree innertype, int nunits, machine_mode mode)
+make_vector_type (tree innertype, poly_int64 nunits, machine_mode mode)
 {
   tree t;
   tree mv_innertype = TYPE_MAIN_VARIANT (innertype);
 
   t = make_node (VECTOR_TYPE);
   TREE_TYPE (t) = mv_innertype;
-  SET_TYPE_VECTOR_SUBPARTS (t, nunits);
+  SET_TYPE_VECTOR_SUBPARTS (t, nunits.to_constant ()); /* Temporary */
   SET_TYPE_MODE (t, mode);
 
   if (TYPE_STRUCTURAL_EQUALITY_P (mv_innertype) || in_lto_p)
@@ -9637,13 +9637,6 @@ make_vector_type (tree innertype, int nunits, machine_mode mode)
   return t;
 }
 
-/* Temporary.  */
-static tree
-make_vector_type (tree innertype, poly_uint64 nunits, machine_mode mode)
-{
-  return make_vector_type (innertype, (int) nunits.to_constant (), mode);
-}
-
 static tree
 make_or_reuse_type (unsigned size, int unsignedp)
 {
@@ -10508,7 +10501,7 @@ reconstruct_complex_type (tree type, tree bottom)
 tree
 build_vector_type_for_mode (tree innertype, machine_mode mode)
 {
-  int nunits;
+  poly_int64 nunits;
   unsigned int bitsize;
 
   switch (GET_MODE_CLASS (mode))
diff --git a/gcc/varasm.c b/gcc/varasm.c
index 8ef91d6..0f5790f 100644
--- a/gcc/varasm.c
+++ b/gcc/varasm.c
@@ -3932,7 +3932,7 @@ output_constant_pool_2 (fixed_size_mode mode, rtx x, unsigned int align)
 	unsigned int subalign = MIN (align, GET_MODE_BITSIZE (submode));
 
 	gcc_assert (GET_CODE (x) == CONST_VECTOR);
-	units = CONST_VECTOR_NUNITS (x);
+	units = GET_MODE_NUNITS (mode);
 
 	for (i = 0; i < units; i++)
 	  {