RISC-V: Remove incorrect function gate gather_scatter_valid_offset_mode_p

Come back to review the codes of gather/scatter, notice gather_scatter_valid_offset_mode_p looks odd.
gather_scatter_valid_offset_mode_p is supposed to block vluxei64/vsuxei64 in RV32 system.
However, it failed to do that since it is passing data_mode instead of index mode:

riscv_vector::gather_scatter_valid_offset_mode_p (<RATIO2:MODE>mode)
It should be RATIO2I instead of RATIO2.
So we have this following iterators which already can block the this situation:

(define_mode_iterator RATIO8I [
  RVVM1QI
  RVVM2HI
  RVVM4SI
  (RVVM8DI "TARGET_VECTOR_ELEN_64 && TARGET_64BIT")
])

We can see TARGET_64BIT to block EEW64 index mode on RV32 system.
So, gather_scatter_valid_offset_mode_p is no longer needed.

After remove it, I find due to incorrect gather_scatter_valid_offset_mode_p.
We failed to vectorize such case in RV32 in the past:

  void __attribute__ ((noinline, noclone))                                     \
  f_##DATA_TYPE (DATA_TYPE *restrict dest, DATA_TYPE *restrict src,            \
		 INDEX##BITS *restrict indices, INDEX##BITS *restrict cond)    \
  {                                                                            \
    for (int i = 0; i < 128; ++i)                                              \
      if (cond[i])                                                             \
	dest[i] += src[indices[i]];                                            \
  }
  T (int64_t, 8)
TEST_ALL (TEST_LOOP)

https://godbolt.org/z/T3ara3fM3

Checked compiler explorer, we can see GCC failed to vectorize it but Clang can vectorize it.

So adapt the tests checking vectorization cases from 8 -> 11.

Confirm we have same behavior as Clang now.

Tested on zvl128/zvl256/zvl512/zvl1024 no regression.

Note this is not an optimization patch, it's buggy codes fix patch.

gcc/ChangeLog:

	* config/riscv/autovec.md
	(mask_len_gather_load<RATIO1:mode><RATIO1:mode>):
	Remove gather_scatter_valid_offset_mode_p.
	(mask_len_gather_load<mode><mode>): Ditto.
	(mask_len_scatter_store<RATIO1:mode><RATIO1:mode>): Ditto.
	(mask_len_scatter_store<mode><mode>): Ditto.
	* config/riscv/predicates.md (const_1_or_8_operand): New predicate.
	(vector_gs_scale_operand_64): Remove.
	* config/riscv/riscv-protos.h (gather_scatter_valid_offset_mode_p): Remove.
	* config/riscv/riscv-v.cc (expand_gather_scatter): Refine code.
	(gather_scatter_valid_offset_mode_p): Remove.
	* config/riscv/vector-iterators.md: Fix iterator bugs.

gcc/testsuite/ChangeLog:

	* gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-1.c: Adapt test.
	* gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-10.c: Ditto.
	* gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-11.c: Ditto.
	* gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-2.c: Ditto.
	* gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-3.c: Ditto.
	* gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-4.c: Ditto.
	* gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-5.c: Ditto.
	* gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-6.c: Ditto.
	* gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-7.c: Ditto.
	* gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-8.c: Ditto.
	* gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-9.c: Ditto.
	* gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-1.c: Ditto.
	* gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-10.c: Ditto.
	* gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-2.c: Ditto.
	* gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-3.c: Ditto.
	* gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-4.c: Ditto.
	* gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-5.c: Ditto.
	* gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-6.c: Ditto.
	* gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-7.c: Ditto.
	* gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-8.c: Ditto.
	* gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-9.c: Ditto.

26 files changed, 65 insertions, 70 deletions

diff --git a/gcc/config/riscv/autovec.md b/gcc/config/riscv/autovec.md
index 80e41af..2d727c2 100644
--- a/gcc/config/riscv/autovec.md
+++ b/gcc/config/riscv/autovec.md
@@ -59,7 +59,7 @@
    (match_operand:<RATIO64:VM> 5 "vector_mask_operand")
    (match_operand 6 "autovec_length_operand")
    (match_operand 7 "const_0_operand")]
-  "TARGET_VECTOR && riscv_vector::gather_scatter_valid_offset_mode_p (<RATIO64:MODE>mode)"
+  "TARGET_VECTOR"
 {
   riscv_vector::expand_gather_scatter (operands, true);
   DONE;
@@ -74,7 +74,7 @@
    (match_operand:<RATIO32:VM> 5 "vector_mask_operand")
    (match_operand 6 "autovec_length_operand")
    (match_operand 7 "const_0_operand")]
-  "TARGET_VECTOR && riscv_vector::gather_scatter_valid_offset_mode_p (<RATIO32:MODE>mode)"
+  "TARGET_VECTOR"
 {
   riscv_vector::expand_gather_scatter (operands, true);
   DONE;
@@ -89,7 +89,7 @@
    (match_operand:<RATIO16:VM> 5 "vector_mask_operand")
    (match_operand 6 "autovec_length_operand")
    (match_operand 7 "const_0_operand")]
-  "TARGET_VECTOR && riscv_vector::gather_scatter_valid_offset_mode_p (<RATIO16:MODE>mode)"
+  "TARGET_VECTOR"
 {
   riscv_vector::expand_gather_scatter (operands, true);
   DONE;
@@ -104,7 +104,7 @@
    (match_operand:<RATIO8:VM> 5 "vector_mask_operand")
    (match_operand 6 "autovec_length_operand")
    (match_operand 7 "const_0_operand")]
-  "TARGET_VECTOR && riscv_vector::gather_scatter_valid_offset_mode_p (<RATIO8:MODE>mode)"
+  "TARGET_VECTOR"
 {
   riscv_vector::expand_gather_scatter (operands, true);
   DONE;
@@ -119,7 +119,7 @@
    (match_operand:<RATIO4:VM> 5 "vector_mask_operand")
    (match_operand 6 "autovec_length_operand")
    (match_operand 7 "const_0_operand")]
-  "TARGET_VECTOR && riscv_vector::gather_scatter_valid_offset_mode_p (<RATIO4:MODE>mode)"
+  "TARGET_VECTOR"
 {
   riscv_vector::expand_gather_scatter (operands, true);
   DONE;
@@ -134,7 +134,7 @@
    (match_operand:<RATIO2:VM> 5 "vector_mask_operand")
    (match_operand 6 "autovec_length_operand")
    (match_operand 7 "const_0_operand")]
-  "TARGET_VECTOR && riscv_vector::gather_scatter_valid_offset_mode_p (<RATIO2:MODE>mode)"
+  "TARGET_VECTOR"
 {
   riscv_vector::expand_gather_scatter (operands, true);
   DONE;
@@ -144,16 +144,16 @@
 ;; larger SEW. Since RVV indexed load/store support zero extend
 ;; implicitly and not support scaling, we should only allow
 ;; operands[3] and operands[4] to be const_1_operand.
-(define_expand "mask_len_gather_load<RATIO1:mode><RATIO1:mode>"
+(define_expand "mask_len_gather_load<mode><mode>"
   [(match_operand:RATIO1 0 "register_operand")
    (match_operand 1 "pmode_reg_or_0_operand")
    (match_operand:RATIO1 2 "register_operand")
-   (match_operand 3 "<RATIO1:gs_extension>")
-   (match_operand 4 "<RATIO1:gs_scale>")
-   (match_operand:<RATIO1:VM> 5 "vector_mask_operand")
+   (match_operand 3 "<gs_extension>")
+   (match_operand 4 "<gs_scale>")
+   (match_operand:<VM> 5 "vector_mask_operand")
    (match_operand 6 "autovec_length_operand")
    (match_operand 7 "const_0_operand")]
-  "TARGET_VECTOR && riscv_vector::gather_scatter_valid_offset_mode_p (<RATIO1:MODE>mode)"
+  "TARGET_VECTOR"
 {
   riscv_vector::expand_gather_scatter (operands, true);
   DONE;
@@ -172,7 +172,7 @@
    (match_operand:<RATIO64:VM> 5 "vector_mask_operand")
    (match_operand 6 "autovec_length_operand")
    (match_operand 7 "const_0_operand")]
-  "TARGET_VECTOR && riscv_vector::gather_scatter_valid_offset_mode_p (<RATIO64:MODE>mode)"
+  "TARGET_VECTOR"
 {
   riscv_vector::expand_gather_scatter (operands, false);
   DONE;
@@ -187,7 +187,7 @@
    (match_operand:<RATIO32:VM> 5 "vector_mask_operand")
    (match_operand 6 "autovec_length_operand")
    (match_operand 7 "const_0_operand")]
-  "TARGET_VECTOR && riscv_vector::gather_scatter_valid_offset_mode_p (<RATIO32:MODE>mode)"
+  "TARGET_VECTOR"
 {
   riscv_vector::expand_gather_scatter (operands, false);
   DONE;
@@ -202,7 +202,7 @@
    (match_operand:<RATIO16:VM> 5 "vector_mask_operand")
    (match_operand 6 "autovec_length_operand")
    (match_operand 7 "const_0_operand")]
-  "TARGET_VECTOR && riscv_vector::gather_scatter_valid_offset_mode_p (<RATIO16:MODE>mode)"
+  "TARGET_VECTOR"
 {
   riscv_vector::expand_gather_scatter (operands, false);
   DONE;
@@ -217,7 +217,7 @@
    (match_operand:<RATIO8:VM> 5 "vector_mask_operand")
    (match_operand 6 "autovec_length_operand")
    (match_operand 7 "const_0_operand")]
-  "TARGET_VECTOR && riscv_vector::gather_scatter_valid_offset_mode_p (<RATIO8:MODE>mode)"
+  "TARGET_VECTOR"
 {
   riscv_vector::expand_gather_scatter (operands, false);
   DONE;
@@ -232,7 +232,7 @@
    (match_operand:<RATIO4:VM> 5 "vector_mask_operand")
    (match_operand 6 "autovec_length_operand")
    (match_operand 7 "const_0_operand")]
-  "TARGET_VECTOR && riscv_vector::gather_scatter_valid_offset_mode_p (<RATIO4:MODE>mode)"
+  "TARGET_VECTOR"
 {
   riscv_vector::expand_gather_scatter (operands, false);
   DONE;
@@ -247,7 +247,7 @@
    (match_operand:<RATIO2:VM> 5 "vector_mask_operand")
    (match_operand 6 "autovec_length_operand")
    (match_operand 7 "const_0_operand")]
-  "TARGET_VECTOR && riscv_vector::gather_scatter_valid_offset_mode_p (<RATIO2:MODE>mode)"
+  "TARGET_VECTOR"
 {
   riscv_vector::expand_gather_scatter (operands, false);
   DONE;
@@ -257,16 +257,16 @@
 ;; larger SEW. Since RVV indexed load/store support zero extend
 ;; implicitly and not support scaling, we should only allow
 ;; operands[3] and operands[4] to be const_1_operand.
-(define_expand "mask_len_scatter_store<RATIO1:mode><RATIO1:mode>"
+(define_expand "mask_len_scatter_store<mode><mode>"
   [(match_operand 0 "pmode_reg_or_0_operand")
    (match_operand:RATIO1 1 "register_operand")
-   (match_operand 2 "<RATIO1:gs_extension>")
-   (match_operand 3 "<RATIO1:gs_scale>")
+   (match_operand 2 "<gs_extension>")
+   (match_operand 3 "<gs_scale>")
    (match_operand:RATIO1 4 "register_operand")
-   (match_operand:<RATIO1:VM> 5 "vector_mask_operand")
+   (match_operand:<VM> 5 "vector_mask_operand")
    (match_operand 6 "autovec_length_operand")
    (match_operand 7 "const_0_operand")]
-  "TARGET_VECTOR && riscv_vector::gather_scatter_valid_offset_mode_p (<RATIO1:MODE>mode)"
+  "TARGET_VECTOR"
 {
   riscv_vector::expand_gather_scatter (operands, false);
   DONE;
diff --git a/gcc/config/riscv/predicates.md b/gcc/config/riscv/predicates.md
index ff213e5..525455f 100644
--- a/gcc/config/riscv/predicates.md
+++ b/gcc/config/riscv/predicates.md
@@ -84,6 +84,10 @@
   (and (match_code "const_int")
        (match_test "INTVAL (op) == 1 || INTVAL (op) == 4")))
 
+(define_predicate "const_1_or_8_operand"
+  (and (match_code "const_int")
+       (match_test "INTVAL (op) == 1 || INTVAL (op) == 8")))
+
 (define_predicate "reg_or_0_operand"
   (ior (match_operand 0 "const_0_operand")
        (match_operand 0 "register_operand")))
@@ -486,10 +490,6 @@
   (ior (match_operand 0 "register_operand")
        (match_code "const_vector")))
 
-(define_predicate "vector_gs_scale_operand_64"
-  (and (match_code "const_int")
-       (match_test "INTVAL (op) == 1 || (INTVAL (op) == 8 && Pmode == DImode)")))
-
 (define_predicate "vector_gs_extension_operand"
   (ior (match_operand 0 "const_1_operand")
        (and (match_operand 0 "const_0_operand")
diff --git a/gcc/config/riscv/riscv-protos.h b/gcc/config/riscv/riscv-protos.h
index c74c2e9..695ee24 100644
--- a/gcc/config/riscv/riscv-protos.h
+++ b/gcc/config/riscv/riscv-protos.h
@@ -594,7 +594,6 @@ opt_machine_mode vectorize_related_mode (machine_mode, scalar_mode,
 unsigned int autovectorize_vector_modes (vec<machine_mode> *, bool);
 bool cmp_lmul_le_one (machine_mode);
 bool cmp_lmul_gt_one (machine_mode);
-bool gather_scatter_valid_offset_mode_p (machine_mode);
 bool vls_mode_valid_p (machine_mode);
 bool vlmax_avl_type_p (rtx_insn *);
 bool has_vl_op (rtx_insn *);
diff --git a/gcc/config/riscv/riscv-v.cc b/gcc/config/riscv/riscv-v.cc
index 52826d0..983c037 100644
--- a/gcc/config/riscv/riscv-v.cc
+++ b/gcc/config/riscv/riscv-v.cc
@@ -3676,7 +3676,7 @@ expand_gather_scatter (rtx *ops, bool is_load)
 	 offset elements.
 
 	 RVV spec only refers to the scale_log == 0 case.  */
-      if (!zero_extend_p || (zero_extend_p && scale_log2 != 0))
+      if (!zero_extend_p || scale_log2 != 0)
 	{
 	  if (zero_extend_p)
 	    inner_idx_mode
@@ -4051,21 +4051,6 @@ vls_mode_valid_p (machine_mode vls_mode)
   return false;
 }
 
-/* Return true if the gather/scatter offset mode is valid.  */
-bool
-gather_scatter_valid_offset_mode_p (machine_mode mode)
-{
-  machine_mode new_mode;
-  /* RISC-V V Spec 18.3:
-     The V extension supports all vector load and store instructions (Section
-     Vector Loads and Stores), except the V extension does not support EEW=64
-     for index values when XLEN=32.  */
-
-  if (GET_MODE_BITSIZE (GET_MODE_INNER (mode)) <= GET_MODE_BITSIZE (Pmode))
-    return get_vector_mode (Pmode, GET_MODE_NUNITS (mode)).exists (&new_mode);
-  return false;
-}
-
 /* We don't have to convert the floating point to integer when the
    mantissa is zero.  Thus, ther will be a limitation for both the
    single and double precision floating point.  There will be no
diff --git a/gcc/config/riscv/vector-iterators.md b/gcc/config/riscv/vector-iterators.md
index 27dae10..56080ed 100644
--- a/gcc/config/riscv/vector-iterators.md
+++ b/gcc/config/riscv/vector-iterators.md
@@ -744,6 +744,17 @@
   (RVVM2DI "TARGET_FULL_V") (RVVM1DI "TARGET_FULL_V")
 ])
 
+;; All RATIO mode iterators are used on gather/scatter vectorization.
+;; RISC-V V Spec 18.3:
+;; The V extension supports all vector load and store instructions (Section
+;; Vector Loads and Stores), except the V extension does not support EEW=64
+;; for index values when XLEN=32.
+;; According to RVV ISA description above, all RATIO index DI mode need TARGET_64BIT.
+;;
+;; In gather/scatter expand, we need to sign/zero extend the index mode into vector
+;; Pmode, so we need to check whether vector Pmode is available.
+;; E.g. when index mode = RVVM8QImde and Pmode = SImode, if it is not zero_extend or
+;; scalar != 1, such gather/scatter is not allowed since we don't have RVVM32SImode.
 (define_mode_iterator RATIO64 [
   (RVVMF8QI "TARGET_MIN_VLEN > 32")
   (RVVMF4HI "TARGET_MIN_VLEN > 32")
@@ -3411,8 +3422,8 @@
 ])
 
 (define_mode_attr gs_extension [
-  (RVVM8QI "const_1_operand") (RVVM4QI "vector_gs_extension_operand")
-  (RVVM2QI "immediate_operand") (RVVM1QI "immediate_operand") (RVVMF2QI "immediate_operand")
+  (RVVM8QI "const_1_operand") (RVVM4QI "const_1_operand")
+  (RVVM2QI "vector_gs_extension_operand") (RVVM1QI "immediate_operand") (RVVMF2QI "immediate_operand")
   (RVVMF4QI "immediate_operand") (RVVMF8QI "immediate_operand")
 
   (RVVM8HI "const_1_operand") (RVVM4HI "vector_gs_extension_operand")
@@ -3455,11 +3466,11 @@
   (RVVM8SF "vector_gs_scale_operand_32_rv32") (RVVM4SF "const_1_or_4_operand") (RVVM2SF "const_1_or_4_operand")
   (RVVM1SF "const_1_or_4_operand") (RVVMF2SF "const_1_or_4_operand")
 
-  (RVVM8DI "vector_gs_scale_operand_64") (RVVM4DI "vector_gs_scale_operand_64")
-  (RVVM2DI "vector_gs_scale_operand_64") (RVVM1DI "vector_gs_scale_operand_64")
+  (RVVM8DI "const_1_or_8_operand") (RVVM4DI "const_1_or_8_operand")
+  (RVVM2DI "const_1_or_8_operand") (RVVM1DI "const_1_or_8_operand")
 
-  (RVVM8DF "vector_gs_scale_operand_64") (RVVM4DF "vector_gs_scale_operand_64")
-  (RVVM2DF "vector_gs_scale_operand_64") (RVVM1DF "vector_gs_scale_operand_64")
+  (RVVM8DF "const_1_or_8_operand") (RVVM4DF "const_1_or_8_operand")
+  (RVVM2DF "const_1_or_8_operand") (RVVM1DF "const_1_or_8_operand")
 ])
 
 (define_int_iterator ORDER [UNSPEC_ORDERED UNSPEC_UNORDERED])
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-1.c
index 055e392..c0b5833 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-1.c
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-1.c
@@ -33,7 +33,7 @@
 
 TEST_ALL (TEST_LOOP)
 
-/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 8 "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 11 "vect" } } */
 /* { dg-final { scan-tree-dump " \.MASK_LEN_GATHER_LOAD" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.GATHER_LOAD" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.MASK_GATHER_LOAD" "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-10.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-10.c
index 5582ca7..9e4fcee 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-10.c
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-10.c
@@ -30,7 +30,7 @@
 
 TEST_ALL (TEST_LOOP)
 
-/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 8 "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 11 "vect" } } */
 /* { dg-final { scan-tree-dump " \.MASK_LEN_GATHER_LOAD" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.GATHER_LOAD" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.MASK_GATHER_LOAD" "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-11.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-11.c
index 54392f4..e574181 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-11.c
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-11.c
@@ -107,7 +107,7 @@ TEST_LOOP (_Float16, uint64_t)
 TEST_LOOP (float, uint64_t)
 TEST_LOOP (double, uint64_t)
 
-/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 64 "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 88 "vect" } } */
 /* { dg-final { scan-tree-dump " \.MASK_LEN_GATHER_LOAD" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.GATHER_LOAD" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.MASK_GATHER_LOAD" "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-2.c
index a2d1358..aadc020 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-2.c
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-2.c
@@ -33,7 +33,7 @@
 
 TEST_ALL (TEST_LOOP)
 
-/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 8 "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 11 "vect" } } */
 /* { dg-final { scan-tree-dump " \.MASK_LEN_GATHER_LOAD" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.GATHER_LOAD" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.MASK_GATHER_LOAD" "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-3.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-3.c
index 66aa5c2..b547dd6 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-3.c
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-3.c
@@ -30,7 +30,7 @@
 
 TEST_ALL (TEST_LOOP)
 
-/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 8 "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 11 "vect" } } */
 /* { dg-final { scan-tree-dump " \.MASK_LEN_GATHER_LOAD" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.GATHER_LOAD" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.MASK_GATHER_LOAD" "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-4.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-4.c
index 80c43de..ac60125 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-4.c
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-4.c
@@ -30,7 +30,7 @@
 
 TEST_ALL (TEST_LOOP)
 
-/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 8 "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 11 "vect" } } */
 /* { dg-final { scan-tree-dump " \.MASK_LEN_GATHER_LOAD" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.GATHER_LOAD" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.MASK_GATHER_LOAD" "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-5.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-5.c
index 972f021..f003a17 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-5.c
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-5.c
@@ -30,7 +30,7 @@
 
 TEST_ALL (TEST_LOOP)
 
-/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 8 "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 11 "vect" } } */
 /* { dg-final { scan-tree-dump " \.MASK_LEN_GATHER_LOAD" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.GATHER_LOAD" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.MASK_GATHER_LOAD" "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-6.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-6.c
index 33114ba..d1d9581 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-6.c
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-6.c
@@ -30,7 +30,7 @@
 
 TEST_ALL (TEST_LOOP)
 
-/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 8 "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 11 "vect" } } */
 /* { dg-final { scan-tree-dump " \.MASK_LEN_GATHER_LOAD" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.GATHER_LOAD" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.MASK_GATHER_LOAD" "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-7.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-7.c
index 729fce0..c8b5cea 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-7.c
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-7.c
@@ -30,7 +30,7 @@
 
 TEST_ALL (TEST_LOOP)
 
-/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 8 "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 11 "vect" } } */
 /* { dg-final { scan-tree-dump " \.MASK_LEN_GATHER_LOAD" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.GATHER_LOAD" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.MASK_GATHER_LOAD" "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-8.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-8.c
index c6f6e88..dfb8a93 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-8.c
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-8.c
@@ -30,7 +30,7 @@
 
 TEST_ALL (TEST_LOOP)
 
-/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 8 "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 11 "vect" } } */
 /* { dg-final { scan-tree-dump " \.MASK_LEN_GATHER_LOAD" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.GATHER_LOAD" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.MASK_GATHER_LOAD" "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-9.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-9.c
index 3629496..32ddba6 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-9.c
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_gather_load_32-9.c
@@ -30,7 +30,7 @@
 
 TEST_ALL (TEST_LOOP)
 
-/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 8 "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 11 "vect" } } */
 /* { dg-final { scan-tree-dump " \.MASK_LEN_GATHER_LOAD" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.GATHER_LOAD" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.MASK_GATHER_LOAD" "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-1.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-1.c
index 5b42776..39e09f4 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-1.c
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-1.c
@@ -33,7 +33,7 @@
 
 TEST_ALL (TEST_LOOP)
 
-/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 8 "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 11 "vect" } } */
 /* { dg-final { scan-tree-dump " \.MASK_LEN_SCATTER_STORE" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.SCATTER_STORE" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.MASK_SCATTER_STORE" "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-10.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-10.c
index d96c96d..b1a2344 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-10.c
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-10.c
@@ -30,7 +30,7 @@
 
 TEST_ALL (TEST_LOOP)
 
-/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 8 "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 11 "vect" } } */
 /* { dg-final { scan-tree-dump " \.MASK_LEN_SCATTER_STORE" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.SCATTER_STORE" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.MASK_SCATTER_STORE" "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-2.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-2.c
index ca939831..b522193 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-2.c
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-2.c
@@ -33,7 +33,7 @@
 
 TEST_ALL (TEST_LOOP)
 
-/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 8 "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 11 "vect" } } */
 /* { dg-final { scan-tree-dump " \.MASK_LEN_SCATTER_STORE" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.SCATTER_STORE" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.MASK_SCATTER_STORE" "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-3.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-3.c
index 60f3a66..4706d0f 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-3.c
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-3.c
@@ -30,7 +30,7 @@
 
 TEST_ALL (TEST_LOOP)
 
-/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 8 "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 11 "vect" } } */
 /* { dg-final { scan-tree-dump " \.MASK_LEN_SCATTER_STORE" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.SCATTER_STORE" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.MASK_SCATTER_STORE" "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-4.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-4.c
index 36f316d..aec7a93 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-4.c
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-4.c
@@ -30,7 +30,7 @@
 
 TEST_ALL (TEST_LOOP)
 
-/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 8 "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 11 "vect" } } */
 /* { dg-final { scan-tree-dump " \.MASK_LEN_SCATTER_STORE" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.SCATTER_STORE" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.MASK_SCATTER_STORE" "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-5.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-5.c
index df58b3d..286e2db 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-5.c
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-5.c
@@ -30,7 +30,7 @@
 
 TEST_ALL (TEST_LOOP)
 
-/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 8 "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 11 "vect" } } */
 /* { dg-final { scan-tree-dump " \.MASK_LEN_SCATTER_STORE" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.SCATTER_STORE" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.MASK_SCATTER_STORE" "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-6.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-6.c
index c2da72a..7674e2a 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-6.c
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-6.c
@@ -30,7 +30,7 @@
 
 TEST_ALL (TEST_LOOP)
 
-/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 8 "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 11 "vect" } } */
 /* { dg-final { scan-tree-dump " \.MASK_LEN_SCATTER_STORE" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.SCATTER_STORE" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.MASK_SCATTER_STORE" "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-7.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-7.c
index e6bdd36..1738f73 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-7.c
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-7.c
@@ -30,7 +30,7 @@
 
 TEST_ALL (TEST_LOOP)
 
-/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 8 "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 11 "vect" } } */
 /* { dg-final { scan-tree-dump " \.MASK_LEN_SCATTER_STORE" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.SCATTER_STORE" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.MASK_SCATTER_STORE" "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-8.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-8.c
index 3ae681d..d819a1a 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-8.c
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-8.c
@@ -30,7 +30,7 @@
 
 TEST_ALL (TEST_LOOP)
 
-/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 8 "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 11 "vect" } } */
 /* { dg-final { scan-tree-dump " \.MASK_LEN_SCATTER_STORE" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.SCATTER_STORE" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.MASK_SCATTER_STORE" "vect" } } */
diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-9.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-9.c
index be5077f..ee453e5 100644
--- a/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-9.c
+++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/gather-scatter/mask_scatter_store_32-9.c
@@ -35,7 +35,7 @@
 
 TEST_ALL (TEST_LOOP)
 
-/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 8 "vect" } } */
+/* { dg-final { scan-tree-dump-times "vectorized 1 loops in function" 11 "vect" } } */
 /* { dg-final { scan-tree-dump " \.MASK_LEN_SCATTER_STORE" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.SCATTER_STORE" "vect" } } */
 /* { dg-final { scan-tree-dump-not " \.MASK_SCATTER_STORE" "vect" } } */