tm.texi: Regenerate.

2012-07-13  Bill Schmidt  <wschmidt@linux.ibm.com>

	* doc/tm.texi: Regenerate.
	* doc/tm.texi.in (TARGET_VECTORIZE_INIT_COST): New hook.
	(TARGET_VECTORIZE_ADD_STMT_COST): Likewise.
	(TARGET_VECTORIZE_FINISH_COST): Likewise.
	(TARGET_VECTORIZE_DESTROY_COST_DATA): Likewise.
	* targhooks.c (default_init_cost): New function.
	(default_add_stmt_cost): Likewise.
	(default_finish_cost): Likewise.
	(default_destroy_cost_data): Likewise.
	* targhooks.h (default_init_cost): New decl.
	(default_add_stmt_cost): Likewise.
	(default_finish_cost): Likewise.
	(default_destroy_cost_data): Likewise.
	* target.def (init_cost): New DEFHOOK.
	(add_stmt_cost): Likewise.
	(finish_cost): Likewise.
	(destroy_cost_data): Likewise.
	* target.h (struct _stmt_vec_info): New extern decl.
	(stmt_vectype): Likewise.
	(stmt_in_inner_loop_p): Likewise.
	* tree-vectorizer.h (stmt_info_for_cost): New struct/typedef.
	(stmt_vector_for_cost): New VEC/typedef.
	(add_stmt_info_to_vec): New function.
	(struct _slp_tree): Remove cost.inside_of_loop field.
	(struct _slp_instance): Remove cost.inside_of_loop field; add
	stmt_cost_vec field.
	(SLP_INSTANCE_INSIDE_OF_LOOP_COST): Remove macro.
	(SLP_INSTANCE_STMT_COST_VEC): New accessor macro.
	(SLP_TREE_INSIDE_OF_LOOP_COST): Remove macro.
	(struct _vect_peel_extended_info): Add stmt_cost_vec field.
	(struct _loop_vec_info): Add target_cost_data field.
	(LOOP_VINFO_TARGET_COST_DATA): New accessor macro.
	(struct _bb_vec_info): Add target_cost_data field.
	(BB_VINFO_TARGET_COST_DATA): New accessor macro.
	(struct _stmt_vec_info): Remove cost.inside_of_loop field.
	(STMT_VINFO_INSIDE_OF_LOOP_COST): Remove macro.
	(stmt_vinfo_set_inside_of_loop_cost): Remove function.
	(init_cost): New function.
	(add_stmt_cost): Likewise.
	(finish_cost): Likewise.
	(destroy_cost_data): Likewise.
	(vect_model_simple_cost): Change parameter list.
	(vect_model_store_cost): Likewise.
	(vect_model_load_cost): Likewise.
	(record_stmt_cost): New extern decl.
	(vect_get_load_cost): Change parameter list.
	(vect_get_store_cost): Likewise.
	* tree-vect-loop.c (new_loop_vec_info): Call init_cost.
	(destroy_loop_vec_info): Call destroy_cost_data.
	(vect_estimate_min_profitable_iters): Remove old calculation of
	inside costs; call finish_cost instead.
	(vect_model_reduction_cost): Call add_stmt_cost instead of old
	inside-costs calculation.
	(vect_model_induction_cost): Likewise.
	* tree-vect-data-refs.c (vect_get_data_access_cost): Change to
	return a stmt_vector_for_cost; modify calls to vect_get_load_cost
	and vect_get_store_cost to obtain the value to return.
	(vect_peeling_hash_get_lowest_cost): Obtain a stmt_cost_vec from
	vect_get_data_access_cost and store it in the minimum peeling
	structure.
	(vect_peeling_hash_choose_best_peeling): Change the parameter list
	to add a (stmt_vector_for_cost *) output parameter, and set its value.
	(vect_enhance_data_refs_alignment): Ignore the new return value from
	calls to vect_get_data_access_cost; obtain stmt_cost_vec from
	vect_peeling_hash_choose_best_peeling and pass its contents to the
	target cost model.
	* tree-vect-stmts.c (stmt_vectype): New function.
	(stmt_in_inner_loop_p): Likewise.
	(record_stmt_cost): Likewise.
	(vect_model_simple_cost): Add stmt_cost_vec parameter; call
	record_stmt_cost instead of old calculation; don't call
	stmt_vinfo_set_inside_of_loop_cost.
	(vect_model_promotion_demotion_cost): Call add_stmt_cost instead of
	old calculation; don't call stmt_vinfo_set_inside_of_loop_cost.
	(vect_model_store_cost): Add stmt_cost_vec parameter; call
	record_stmt_cost instead of old calculation; add stmt_cost_vec
	parameter to vect_get_store_cost call; don't call
	stmt_vinfo_set_inside_of_loop_cost.
	(vect_get_store_cost): Add stmt_cost_vec parameter; call
	record_stmt_cost instead of old calculation.
	(vect_model_load_cost): Add stmt_cost_vec parameter; call
	record_stmt_cost instead of old calculation; add stmt_cost_vec
	parameter to vect_get_load_cost call; don't call
	stmt_vinfo_set_inside_of_loop_cost.
	(vect_get_load_cost): Add stmt_cost_vec parameter; call
	record_stmt_cost instead of old calculation.
	(vectorizable_call): Add NULL parameter to vect_model_simple_cost call.
	(vectorizable_conversion): Likewise.
	(vectorizable_assignment): Likewise.
	(vectorizable_shift): Likewise.
	(vectorizable_operation): Likewise.
	(vectorizable_store): Add NULL parameter to vect_model_store_cost call.
	(vectorizable_load): Add NULL parameter to vect_model_load_cost call.
	(new_stmt_vec_info): Don't set STMT_VINFO_INSIDE_OF_LOOP_COST.
	* config/spu/spu.c (TARGET_VECTORIZE_INIT_COST): New macro def.
	(TARGET_VECTORIZE_ADD_STMT_COST): Likewise.
	(TARGET_VECTORIZE_FINISH_COST): Likewise.
	(TARGET_VECTORIZE_DESTROY_COST_DATA): Likewise.
	(spu_init_cost): New function.
	(spu_add_stmt_cost): Likewise.
	(spu_finish_cost): Likewise.
	(spu_destroy_cost_data): Likewise.
	* config/i386/i386.c (ix86_init_cost): New function.
	(ix86_add_stmt_cost): Likewise.
	(ix86_finish_cost): Likewise.
	(ix86_destroy_cost_data): Likewise.
	(TARGET_VECTORIZE_INIT_COST): New macro def.
	(TARGET_VECTORIZE_ADD_STMT_COST): Likewise.
	(TARGET_VECTORIZE_FINISH_COST): Likewise.
	(TARGET_VECTORIZE_DESTROY_COST_DATA): Likewise.
	* config/rs6000/rs6000.c (TARGET_VECTORIZE_INIT_COST): New macro def.
	(TARGET_VECTORIZE_ADD_STMT_COST): Likewise.
	(TARGET_VECTORIZE_FINISH_COST): Likewise.
	(TARGET_VECTORIZE_DESTROY_COST_DATA): Likewise.
	(rs6000_init_cost): New function.
	(rs6000_add_stmt_cost): Likewise.
	(rs6000_finish_cost): Likewise.
	(rs6000_destroy_cost_data): Likewise.
	* tree-vect-slp.c (vect_free_slp_instance): Free stmt_cost_vec.
	(vect_create_new_slp_node): Don't set SLP_TREE_INSIDE_OF_LOOP_COST.
	(vect_get_and_check_slp_defs): Add stmt_cost_vec parameter; add
	stmt_cost_vec parameter to vect_model_store_cost and
	vect_model_simple_cost calls.
	(vect_build_slp_tree): Remove inside_cost parameter; add stmt_cost_vec
	parameter; add stmt_cost_vec parameter to vect_get_and_check_slp_defs,
	vect_model_load_cost, and recursive vect_build_slp_tree calls; prevent
	calculating cost more than once for loads; remove inside_cost
	parameter from recursive vect_build_slp_tree calls; call
	record_stmt_cost instead of old calculation.
	(vect_analyze_slp_instance): Allocate stmt_cost_vec and save it with
	the instance; free it on premature exit; remove inside_cost parameter
	from vect_build_slp_tree call; add stmt_cost_vec parameter to
	vect_build_slp_tree call; don't set SLP_INSTANCE_INSIDE_OF_LOOP_COST.
	(new_bb_vec_info): Call init_cost.
	(destroy_bb_vec_info): Call destroy_cost_data.
	(vect_bb_vectorization_profitable_p): Call add_stmt_cost for each
	statement recorded with an SLP instance; call finish_cost instead of
	the old calculation.
	(vect_update_slp_costs_according_to_vf): Record statement costs from
	SLP instances, multiplying by the appropriate number of copies; don't
	update SLP_INSTANCE_INSIDE_OF_LOOP_COST.

From-SVN: r189460

1 files changed, 92 insertions, 29 deletions

diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h
index 4c5ea36..6b74bb4 100644
--- a/gcc/tree-vectorizer.h
+++ b/gcc/tree-vectorizer.h
@@ -71,6 +71,32 @@ enum vect_def_type {
                                    || ((D) == vect_double_reduction_def) \
                                    || ((D) == vect_nested_cycle))
 
+/* Structure to encapsulate information about a group of like
+   instructions to be presented to the target cost model.  */
+typedef struct _stmt_info_for_cost {
+  int count;
+  enum vect_cost_for_stmt kind;
+  gimple stmt;
+  int misalign;
+} stmt_info_for_cost;
+
+DEF_VEC_O (stmt_info_for_cost);
+DEF_VEC_ALLOC_O (stmt_info_for_cost, heap);
+
+typedef VEC(stmt_info_for_cost, heap) *stmt_vector_for_cost;
+
+static inline void
+add_stmt_info_to_vec (stmt_vector_for_cost *stmt_cost_vec, int count,
+		      enum vect_cost_for_stmt kind, gimple stmt, int misalign)
+{
+  stmt_info_for_cost si;
+  si.count = count;
+  si.kind = kind;
+  si.stmt = stmt;
+  si.misalign = misalign;
+  VEC_safe_push (stmt_info_for_cost, heap, *stmt_cost_vec, &si);
+}
+
 /************************************************************************
   SLP
  ************************************************************************/
@@ -96,7 +122,6 @@ typedef struct _slp_tree {
   struct
   {
     int outside_of_loop;     /* Statements generated outside loop.  */
-    int inside_of_loop;      /* Statements generated inside loop.  */
   } cost;
 } *slp_tree;
 
@@ -119,9 +144,11 @@ typedef struct _slp_instance {
   struct
   {
     int outside_of_loop;     /* Statements generated outside loop.  */
-    int inside_of_loop;      /* Statements generated inside loop.  */
   } cost;
 
+  /* Inside-loop costs.  */
+  stmt_vector_for_cost stmt_cost_vec;
+
   /* Loads permutation relatively to the stores, NULL if there is no
      permutation.  */
   VEC (int, heap) *load_permutation;
@@ -142,7 +169,7 @@ DEF_VEC_ALLOC_P(slp_instance, heap);
 #define SLP_INSTANCE_GROUP_SIZE(S)               (S)->group_size
 #define SLP_INSTANCE_UNROLLING_FACTOR(S)         (S)->unrolling_factor
 #define SLP_INSTANCE_OUTSIDE_OF_LOOP_COST(S)     (S)->cost.outside_of_loop
-#define SLP_INSTANCE_INSIDE_OF_LOOP_COST(S)      (S)->cost.inside_of_loop
+#define SLP_INSTANCE_STMT_COST_VEC(S)            (S)->stmt_cost_vec
 #define SLP_INSTANCE_LOAD_PERMUTATION(S)         (S)->load_permutation
 #define SLP_INSTANCE_LOADS(S)                    (S)->loads
 #define SLP_INSTANCE_FIRST_LOAD_STMT(S)          (S)->first_load
@@ -152,7 +179,6 @@ DEF_VEC_ALLOC_P(slp_instance, heap);
 #define SLP_TREE_VEC_STMTS(S)                    (S)->vec_stmts
 #define SLP_TREE_NUMBER_OF_VEC_STMTS(S)          (S)->vec_stmts_size
 #define SLP_TREE_OUTSIDE_OF_LOOP_COST(S)         (S)->cost.outside_of_loop
-#define SLP_TREE_INSIDE_OF_LOOP_COST(S)          (S)->cost.inside_of_loop
 
 /* This structure is used in creation of an SLP tree.  Each instance
    corresponds to the same operand in a group of scalar stmts in an SLP
@@ -186,6 +212,7 @@ typedef struct _vect_peel_extended_info
   struct _vect_peel_info peel_info;
   unsigned int inside_cost;
   unsigned int outside_cost;
+  stmt_vector_for_cost stmt_cost_vec;
 } *vect_peel_extended_info;
 
 /*-----------------------------------------------------------------*/
@@ -274,6 +301,9 @@ typedef struct _loop_vec_info {
   /* Hash table used to choose the best peeling option.  */
   htab_t peeling_htab;
 
+  /* Cost data used by the target cost model.  */
+  void *target_cost_data;
+
   /* When we have grouped data accesses with gaps, we may introduce invalid
      memory accesses.  We peel the last iteration of the loop to prevent
      this.  */
@@ -307,6 +337,7 @@ typedef struct _loop_vec_info {
 #define LOOP_VINFO_REDUCTIONS(L)           (L)->reductions
 #define LOOP_VINFO_REDUCTION_CHAINS(L)     (L)->reduction_chains
 #define LOOP_VINFO_PEELING_HTAB(L)         (L)->peeling_htab
+#define LOOP_VINFO_TARGET_COST_DATA(L)     (L)->target_cost_data
 #define LOOP_VINFO_PEELING_FOR_GAPS(L)     (L)->peeling_for_gaps
 
 #define LOOP_REQUIRES_VERSIONING_FOR_ALIGNMENT(L) \
@@ -350,13 +381,18 @@ typedef struct _bb_vec_info {
 
   /* All data dependences in the basic block.  */
   VEC (ddr_p, heap) *ddrs;
+
+  /* Cost data used by the target cost model.  */
+  void *target_cost_data;
+
 } *bb_vec_info;
 
-#define BB_VINFO_BB(B)              (B)->bb
-#define BB_VINFO_GROUPED_STORES(B)  (B)->grouped_stores
-#define BB_VINFO_SLP_INSTANCES(B)   (B)->slp_instances
-#define BB_VINFO_DATAREFS(B)        (B)->datarefs
-#define BB_VINFO_DDRS(B)            (B)->ddrs
+#define BB_VINFO_BB(B)               (B)->bb
+#define BB_VINFO_GROUPED_STORES(B)   (B)->grouped_stores
+#define BB_VINFO_SLP_INSTANCES(B)    (B)->slp_instances
+#define BB_VINFO_DATAREFS(B)         (B)->datarefs
+#define BB_VINFO_DDRS(B)             (B)->ddrs
+#define BB_VINFO_TARGET_COST_DATA(B) (B)->target_cost_data
 
 static inline bb_vec_info
 vec_info_for_bb (basic_block bb)
@@ -534,7 +570,6 @@ typedef struct _stmt_vec_info {
   struct
   {
     int outside_of_loop;     /* Statements generated outside loop.  */
-    int inside_of_loop;      /* Statements generated inside loop.  */
   } cost;
 
   /* The bb_vec_info with respect to which STMT is vectorized.  */
@@ -594,7 +629,6 @@ typedef struct _stmt_vec_info {
 
 #define STMT_VINFO_RELEVANT_P(S)          ((S)->relevant != vect_unused_in_scope)
 #define STMT_VINFO_OUTSIDE_OF_LOOP_COST(S) (S)->cost.outside_of_loop
-#define STMT_VINFO_INSIDE_OF_LOOP_COST(S)  (S)->cost.inside_of_loop
 
 #define HYBRID_SLP_STMT(S)                ((S)->slp_type == hybrid)
 #define PURE_SLP_STMT(S)                  ((S)->slp_type == pure_slp)
@@ -733,19 +767,7 @@ is_loop_header_bb_p (basic_block bb)
   return false;
 }
 
-/* Set inside loop vectorization cost.  */
-
-static inline void
-stmt_vinfo_set_inside_of_loop_cost (stmt_vec_info stmt_info, slp_tree slp_node,
-				    int cost)
-{
-  if (slp_node)
-    SLP_TREE_INSIDE_OF_LOOP_COST (slp_node) = cost;
-  else
-    STMT_VINFO_INSIDE_OF_LOOP_COST (stmt_info) = cost;
-}
-
-/* Set inside loop vectorization cost.  */
+/* Set outside loop vectorization cost.  */
 
 static inline void
 stmt_vinfo_set_outside_of_loop_cost (stmt_vec_info stmt_info, slp_tree slp_node,
@@ -782,6 +804,41 @@ int vect_get_stmt_cost (enum vect_cost_for_stmt type_of_cost)
                                                        dummy_type, dummy);
 }
 
+/* Alias targetm.vectorize.init_cost.  */
+
+static inline void *
+init_cost (struct loop *loop_info)
+{
+  return targetm.vectorize.init_cost (loop_info);
+}
+
+/* Alias targetm.vectorize.add_stmt_cost.  */
+
+static inline unsigned
+add_stmt_cost (void *data, int count, enum vect_cost_for_stmt kind,
+	       stmt_vec_info stmt_info, int misalign)
+{
+  return targetm.vectorize.add_stmt_cost (data, count, kind,
+					  stmt_info, misalign);
+}
+
+/* Alias targetm.vectorize.finish_cost.  */
+
+static inline unsigned
+finish_cost (void *data)
+{
+  return targetm.vectorize.finish_cost (data);
+}
+
+/* Alias targetm.vectorize.destroy_cost_data.  */
+
+static inline void
+destroy_cost_data (void *data)
+{
+  targetm.vectorize.destroy_cost_data (data);
+}
+
+
 /*-----------------------------------------------------------------*/
 /* Info on data references alignment.                              */
 /*-----------------------------------------------------------------*/
@@ -849,10 +906,14 @@ extern stmt_vec_info new_stmt_vec_info (gimple stmt, loop_vec_info,
 extern void free_stmt_vec_info (gimple stmt);
 extern tree vectorizable_function (gimple, tree, tree);
 extern void vect_model_simple_cost (stmt_vec_info, int, enum vect_def_type *,
-                                    slp_tree);
+                                    slp_tree, stmt_vector_for_cost *);
 extern void vect_model_store_cost (stmt_vec_info, int, bool,
-				   enum vect_def_type, slp_tree);
-extern void vect_model_load_cost (stmt_vec_info, int, bool, slp_tree);
+				   enum vect_def_type, slp_tree,
+				   stmt_vector_for_cost *);
+extern void vect_model_load_cost (stmt_vec_info, int, bool, slp_tree,
+				  stmt_vector_for_cost *);
+extern unsigned record_stmt_cost (stmt_vector_for_cost *, int,
+				  enum vect_cost_for_stmt, stmt_vec_info, int);
 extern void vect_finish_stmt_generation (gimple, gimple,
                                          gimple_stmt_iterator *);
 extern bool vect_mark_stmts_to_be_vectorized (loop_vec_info);
@@ -867,8 +928,10 @@ extern bool vect_analyze_stmt (gimple, bool *, slp_tree);
 extern bool vectorizable_condition (gimple, gimple_stmt_iterator *, gimple *,
                                     tree, int, slp_tree);
 extern void vect_get_load_cost (struct data_reference *, int, bool,
-                                unsigned int *, unsigned int *);
-extern void vect_get_store_cost (struct data_reference *, int, unsigned int *);
+				unsigned int *, unsigned int *,
+				stmt_vector_for_cost *);
+extern void vect_get_store_cost (struct data_reference *, int,
+				 unsigned int *, stmt_vector_for_cost *);
 extern bool vect_supportable_shift (enum tree_code, tree);
 extern void vect_get_vec_defs (tree, tree, gimple, VEC (tree, heap) **,
 			       VEC (tree, heap) **, slp_tree, int);