From 68326d5d1a593dc0bf098c03aac25916168bc5a9 Mon Sep 17 00:00:00 2001
From: Alex Coplan <alex.coplan@arm.com>
Date: Mon, 11 Mar 2024 13:09:10 +0000
Subject: vect: Force alignment peeling to vectorize more early break loops
 [PR118211]

This allows us to vectorize more loops with early exits by forcing
peeling for alignment to make sure that we're guaranteed to be able to
safely read an entire vector iteration without crossing a page boundary.

To make this work for VLA architectures we have to allow compile-time
non-constant target alignments.  We also have to override the result of
the target's preferred_vector_alignment hook if it isn't a power-of-two
multiple of the TYPE_SIZE of the chosen vector type.

gcc/ChangeLog:

	PR tree-optimization/118211
	PR tree-optimization/116126
	* tree-vect-data-refs.cc (vect_analyze_early_break_dependences):
	Set need_peeling_for_alignment flag on read DRs instead of
	failing vectorization.  Punt on gathers.
	(dr_misalignment): Handle non-constant target alignments.
	(vect_compute_data_ref_alignment): If need_peeling_for_alignment
	flag is set on the DR, then override the target alignment chosen
	by the preferred_vector_alignment hook to choose a safe
	alignment.
	(vect_supportable_dr_alignment): Override
	support_vector_misalignment hook if need_peeling_for_alignment
	is set on the DR: in this case we must return
	dr_unaligned_unsupported in order to force peeling.
	* tree-vect-loop-manip.cc (vect_do_peeling): Allow prolog
	peeling by a compile-time non-constant amount.
	* tree-vectorizer.h (dr_vec_info): Add new flag
	need_peeling_for_alignment.

gcc/testsuite/ChangeLog:

	PR tree-optimization/118211
	PR tree-optimization/116126
	* gcc.dg/tree-ssa/cunroll-13.c: Don't vectorize.
	* gcc.dg/tree-ssa/cunroll-14.c: Likewise.
	* gcc.dg/unroll-6.c: Likewise.
	* gcc.dg/tree-ssa/gen-vect-28.c: Likewise.
	* gcc.dg/vect/vect-104.c: Expect to vectorize.
	* gcc.dg/vect/vect-early-break_108-pr113588.c: Likewise.
	* gcc.dg/vect/vect-early-break_109-pr113588.c: Likewise.
	* gcc.dg/vect/vect-early-break_110-pr113467.c: Likewise.
	* gcc.dg/vect/vect-early-break_3.c: Likewise.
	* gcc.dg/vect/vect-early-break_65.c: Likewise.
	* gcc.dg/vect/vect-early-break_8.c: Likewise.
	* gfortran.dg/vect/vect-5.f90: Likewise.
	* gfortran.dg/vect/vect-8.f90: Likewise.
	* gcc.dg/vect/vect-switch-search-line-fast.c:

Co-Authored-By: Tamar Christina <tamar.christina@arm.com>
---
 gcc/tree-vectorizer.h | 5 +++++
 1 file changed, 5 insertions(+)

(limited to 'gcc/tree-vectorizer.h')

diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h
index 135eb11..79db02a 100644
--- a/gcc/tree-vectorizer.h
+++ b/gcc/tree-vectorizer.h
@@ -1278,6 +1278,11 @@ public:
   poly_uint64 target_alignment;
   /* If true the alignment of base_decl needs to be increased.  */
   bool base_misaligned;
+
+  /* Set by early break vectorization when this DR needs peeling for alignment
+     for correctness.  */
+  bool need_peeling_for_alignment;
+
   tree base_decl;
 
   /* Stores current vectorized loop's offset.  To be added to the DR's
-- 
cgit v1.1