[vect]PR 88915: Vectorize epilogues when versioning loops

gcc/ChangeLog:
2019-10-29  Andre Vieira  <andre.simoesdiasvieira@arm.com>

	PR 88915
	* tree-ssa-loop-niter.h (simplify_replace_tree): Change declaration.
	* tree-ssa-loop-niter.c (simplify_replace_tree): Add context parameter
	and make the valueize function pointer also take a void pointer.
	* gcc/tree-ssa-sccvn.c (vn_valueize_wrapper): New function to wrap
	around vn_valueize, to call it without a context.
	(process_bb): Use vn_valueize_wrapper instead of vn_valueize.
	* tree-vect-loop.c (_loop_vec_info): Initialize epilogue_vinfos.
	(~_loop_vec_info): Release epilogue_vinfos.
	(vect_analyze_loop_costing): Use knowledge of main VF to estimate
	number of iterations of epilogue.
	(vect_analyze_loop_2): Adapt to analyse main loop for all supported
	vector sizes when vect-epilogues-nomask=1.  Also keep track of lowest
	versioning threshold needed for main loop.
	(vect_analyze_loop): Likewise.
	(find_in_mapping): New helper function.
	(update_epilogue_loop_vinfo): New function.
	(vect_transform_loop): When vectorizing epilogues re-use analysis done
	on main loop and call update_epilogue_loop_vinfo to update it.
	* tree-vect-loop-manip.c (vect_update_inits_of_drs): No longer insert
	stmts on loop preheader edge.
	(vect_do_peeling): Enable skip-vectors when doing loop versioning if
	we decided to vectorize epilogues.  Update epilogues NITERS and
	construct ADVANCE to update epilogues data references where needed.
	* tree-vectorizer.h (_loop_vec_info): Add epilogue_vinfos.
	(vect_do_peeling, vect_update_inits_of_drs,
	 determine_peel_for_niter, vect_analyze_loop): Add or update
	declarations.
	* tree-vectorizer.c (try_vectorize_loop_1): Make sure to use already
	created loop_vec_info's for epilogues when available.  Otherwise analyse
	epilogue separately.

From-SVN: r277569

1 files changed, 12 insertions, 1 deletions

diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h
index 56be28b..71b5f38 100644
--- a/gcc/tree-vectorizer.h
+++ b/gcc/tree-vectorizer.h
@@ -26,6 +26,7 @@ typedef class _stmt_vec_info *stmt_vec_info;
 #include "tree-data-ref.h"
 #include "tree-hash-traits.h"
 #include "target.h"
+#include <utility>
 
 /* Used for naming of new temporaries.  */
 enum vect_var_kind {
@@ -456,6 +457,8 @@ struct rgroup_masks {
 
 typedef auto_vec<rgroup_masks> vec_loop_masks;
 
+typedef auto_vec<std::pair<data_reference*, tree> > drs_init_vec;
+
 /*-----------------------------------------------------------------*/
 /* Info on vectorized loops.                                       */
 /*-----------------------------------------------------------------*/
@@ -639,6 +642,10 @@ public:
      this points to the original vectorized loop.  Otherwise NULL.  */
   _loop_vec_info *orig_loop_info;
 
+  /* Used to store loop_vec_infos of epilogues of this loop during
+     analysis.  */
+  vec<_loop_vec_info *> epilogue_vinfos;
+
 } *loop_vec_info;
 
 /* Access Functions.  */
@@ -1589,10 +1596,12 @@ class loop *slpeel_tree_duplicate_loop_to_edge_cfg (class loop *,
 						     class loop *, edge);
 class loop *vect_loop_versioning (loop_vec_info);
 extern class loop *vect_do_peeling (loop_vec_info, tree, tree,
-				     tree *, tree *, tree *, int, bool, bool);
+				    tree *, tree *, tree *, int, bool, bool,
+				    tree *, drs_init_vec &);
 extern void vect_prepare_for_masked_peels (loop_vec_info);
 extern dump_user_location_t find_loop_location (class loop *);
 extern bool vect_can_advance_ivs_p (loop_vec_info);
+extern void vect_update_inits_of_drs (loop_vec_info, tree, tree_code);
 
 /* In tree-vect-stmts.c.  */
 extern tree get_vectype_for_scalar_type (vec_info *, tree);
@@ -1700,6 +1709,8 @@ extern tree vect_create_addr_base_for_vector_ref (stmt_vec_info, gimple_seq *,
 
 /* In tree-vect-loop.c.  */
 extern widest_int vect_iv_limit_for_full_masking (loop_vec_info loop_vinfo);
+/* Used in tree-vect-loop-manip.c */
+extern void determine_peel_for_niter (loop_vec_info);
 /* Used in gimple-loop-interchange.c and tree-parloops.c.  */
 extern bool check_reduction_path (dump_user_location_t, loop_p, gphi *, tree,
 				  enum tree_code);