tree-optimization/101801 - rework generic vector vectorization more

This builds ontop of the vect_worthwhile_without_simd_p refactoring
done earlier.  It was wrong in dropping the appearant double checks
for operation support since the optab check can happen with an
integer vector emulation mode and thus succeed but vector lowering
might not actually support the operation on word_mode.

The following patch adds a vect_emulated_vector_p helper and
re-instantiates the check where it was previously.  It also adds
appropriate costing of the scalar stmts emitted by vector lowering
to vectorizable_operation which should be the only place such
operations are synthesized.  I've also cared for the case where
the vector mode is supported but the operation is not (though
I think this will be unlikely given we're talking about plus, minus
and negate).

This fixes the observed FAIL of gcc.dg/tree-ssa/gen-vect-11b.c
with -m32 where we end up vectorizing a multiplication that ends up
being teared down to scalars again by vector lowering.

I'm not super happy about all the other places where we're now
and previously feeding scalar modes to optab checks where we
want to know whether we can vectorize sth but well.

2021-09-08  Richard Biener  <rguenther@suse.de>

	PR tree-optimization/101801
	PR tree-optimization/101819
	* tree-vectorizer.h (vect_emulated_vector_p): Declare.
	* tree-vect-loop.c (vect_emulated_vector_p): New function.
	(vectorizable_reduction): Re-instantiate a check for emulated
	operations.
	* tree-vect-stmts.c (vectorizable_shift): Likewise.
	(vectorizable_operation): Likewise.  Cost emulated vector
	operations according to the scalar sequence synthesized by
	vector lowering.

1 files changed, 18 insertions, 0 deletions

diff --git a/gcc/tree-vect-loop.c b/gcc/tree-vect-loop.c
index 37c7daa..995d143 100644
--- a/gcc/tree-vect-loop.c
+++ b/gcc/tree-vect-loop.c
@@ -7234,6 +7234,14 @@ vectorizable_reduction (loop_vec_info loop_vinfo,
 	      dump_printf (MSG_NOTE, "proceeding using word mode.\n");
         }
 
+      if (vect_emulated_vector_p (vectype_in)
+	  && !vect_can_vectorize_without_simd_p (code))
+	{
+	  if (dump_enabled_p ())
+	    dump_printf (MSG_NOTE, "using word mode not possible.\n");
+	  return false;
+	}
+
       /* lane-reducing operations have to go through vect_transform_reduction.
          For the other cases try without the single cycle optimization.  */
       if (!ok)
@@ -7936,6 +7944,16 @@ vectorizable_phi (vec_info *,
   return true;
 }
 
+/* Return true if VECTYPE represents a vector that requires lowering
+   by the vector lowering pass.  */
+
+bool
+vect_emulated_vector_p (tree vectype)
+{
+  return (!VECTOR_MODE_P (TYPE_MODE (vectype))
+	  && (!VECTOR_BOOLEAN_TYPE_P (vectype)
+	      || TYPE_PRECISION (TREE_TYPE (vectype)) != 1));
+}
 
 /* Return true if we can emulate CODE on an integer mode representation
    of a vector.  */