tm.texi: Regenerate.

2012-07-13  Bill Schmidt  <wschmidt@linux.ibm.com>

	* doc/tm.texi: Regenerate.
	* doc/tm.texi.in (TARGET_VECTORIZE_INIT_COST): New hook.
	(TARGET_VECTORIZE_ADD_STMT_COST): Likewise.
	(TARGET_VECTORIZE_FINISH_COST): Likewise.
	(TARGET_VECTORIZE_DESTROY_COST_DATA): Likewise.
	* targhooks.c (default_init_cost): New function.
	(default_add_stmt_cost): Likewise.
	(default_finish_cost): Likewise.
	(default_destroy_cost_data): Likewise.
	* targhooks.h (default_init_cost): New decl.
	(default_add_stmt_cost): Likewise.
	(default_finish_cost): Likewise.
	(default_destroy_cost_data): Likewise.
	* target.def (init_cost): New DEFHOOK.
	(add_stmt_cost): Likewise.
	(finish_cost): Likewise.
	(destroy_cost_data): Likewise.
	* target.h (struct _stmt_vec_info): New extern decl.
	(stmt_vectype): Likewise.
	(stmt_in_inner_loop_p): Likewise.
	* tree-vectorizer.h (stmt_info_for_cost): New struct/typedef.
	(stmt_vector_for_cost): New VEC/typedef.
	(add_stmt_info_to_vec): New function.
	(struct _slp_tree): Remove cost.inside_of_loop field.
	(struct _slp_instance): Remove cost.inside_of_loop field; add
	stmt_cost_vec field.
	(SLP_INSTANCE_INSIDE_OF_LOOP_COST): Remove macro.
	(SLP_INSTANCE_STMT_COST_VEC): New accessor macro.
	(SLP_TREE_INSIDE_OF_LOOP_COST): Remove macro.
	(struct _vect_peel_extended_info): Add stmt_cost_vec field.
	(struct _loop_vec_info): Add target_cost_data field.
	(LOOP_VINFO_TARGET_COST_DATA): New accessor macro.
	(struct _bb_vec_info): Add target_cost_data field.
	(BB_VINFO_TARGET_COST_DATA): New accessor macro.
	(struct _stmt_vec_info): Remove cost.inside_of_loop field.
	(STMT_VINFO_INSIDE_OF_LOOP_COST): Remove macro.
	(stmt_vinfo_set_inside_of_loop_cost): Remove function.
	(init_cost): New function.
	(add_stmt_cost): Likewise.
	(finish_cost): Likewise.
	(destroy_cost_data): Likewise.
	(vect_model_simple_cost): Change parameter list.
	(vect_model_store_cost): Likewise.
	(vect_model_load_cost): Likewise.
	(record_stmt_cost): New extern decl.
	(vect_get_load_cost): Change parameter list.
	(vect_get_store_cost): Likewise.
	* tree-vect-loop.c (new_loop_vec_info): Call init_cost.
	(destroy_loop_vec_info): Call destroy_cost_data.
	(vect_estimate_min_profitable_iters): Remove old calculation of
	inside costs; call finish_cost instead.
	(vect_model_reduction_cost): Call add_stmt_cost instead of old
	inside-costs calculation.
	(vect_model_induction_cost): Likewise.
	* tree-vect-data-refs.c (vect_get_data_access_cost): Change to
	return a stmt_vector_for_cost; modify calls to vect_get_load_cost
	and vect_get_store_cost to obtain the value to return.
	(vect_peeling_hash_get_lowest_cost): Obtain a stmt_cost_vec from
	vect_get_data_access_cost and store it in the minimum peeling
	structure.
	(vect_peeling_hash_choose_best_peeling): Change the parameter list
	to add a (stmt_vector_for_cost *) output parameter, and set its value.
	(vect_enhance_data_refs_alignment): Ignore the new return value from
	calls to vect_get_data_access_cost; obtain stmt_cost_vec from
	vect_peeling_hash_choose_best_peeling and pass its contents to the
	target cost model.
	* tree-vect-stmts.c (stmt_vectype): New function.
	(stmt_in_inner_loop_p): Likewise.
	(record_stmt_cost): Likewise.
	(vect_model_simple_cost): Add stmt_cost_vec parameter; call
	record_stmt_cost instead of old calculation; don't call
	stmt_vinfo_set_inside_of_loop_cost.
	(vect_model_promotion_demotion_cost): Call add_stmt_cost instead of
	old calculation; don't call stmt_vinfo_set_inside_of_loop_cost.
	(vect_model_store_cost): Add stmt_cost_vec parameter; call
	record_stmt_cost instead of old calculation; add stmt_cost_vec
	parameter to vect_get_store_cost call; don't call
	stmt_vinfo_set_inside_of_loop_cost.
	(vect_get_store_cost): Add stmt_cost_vec parameter; call
	record_stmt_cost instead of old calculation.
	(vect_model_load_cost): Add stmt_cost_vec parameter; call
	record_stmt_cost instead of old calculation; add stmt_cost_vec
	parameter to vect_get_load_cost call; don't call
	stmt_vinfo_set_inside_of_loop_cost.
	(vect_get_load_cost): Add stmt_cost_vec parameter; call
	record_stmt_cost instead of old calculation.
	(vectorizable_call): Add NULL parameter to vect_model_simple_cost call.
	(vectorizable_conversion): Likewise.
	(vectorizable_assignment): Likewise.
	(vectorizable_shift): Likewise.
	(vectorizable_operation): Likewise.
	(vectorizable_store): Add NULL parameter to vect_model_store_cost call.
	(vectorizable_load): Add NULL parameter to vect_model_load_cost call.
	(new_stmt_vec_info): Don't set STMT_VINFO_INSIDE_OF_LOOP_COST.
	* config/spu/spu.c (TARGET_VECTORIZE_INIT_COST): New macro def.
	(TARGET_VECTORIZE_ADD_STMT_COST): Likewise.
	(TARGET_VECTORIZE_FINISH_COST): Likewise.
	(TARGET_VECTORIZE_DESTROY_COST_DATA): Likewise.
	(spu_init_cost): New function.
	(spu_add_stmt_cost): Likewise.
	(spu_finish_cost): Likewise.
	(spu_destroy_cost_data): Likewise.
	* config/i386/i386.c (ix86_init_cost): New function.
	(ix86_add_stmt_cost): Likewise.
	(ix86_finish_cost): Likewise.
	(ix86_destroy_cost_data): Likewise.
	(TARGET_VECTORIZE_INIT_COST): New macro def.
	(TARGET_VECTORIZE_ADD_STMT_COST): Likewise.
	(TARGET_VECTORIZE_FINISH_COST): Likewise.
	(TARGET_VECTORIZE_DESTROY_COST_DATA): Likewise.
	* config/rs6000/rs6000.c (TARGET_VECTORIZE_INIT_COST): New macro def.
	(TARGET_VECTORIZE_ADD_STMT_COST): Likewise.
	(TARGET_VECTORIZE_FINISH_COST): Likewise.
	(TARGET_VECTORIZE_DESTROY_COST_DATA): Likewise.
	(rs6000_init_cost): New function.
	(rs6000_add_stmt_cost): Likewise.
	(rs6000_finish_cost): Likewise.
	(rs6000_destroy_cost_data): Likewise.
	* tree-vect-slp.c (vect_free_slp_instance): Free stmt_cost_vec.
	(vect_create_new_slp_node): Don't set SLP_TREE_INSIDE_OF_LOOP_COST.
	(vect_get_and_check_slp_defs): Add stmt_cost_vec parameter; add
	stmt_cost_vec parameter to vect_model_store_cost and
	vect_model_simple_cost calls.
	(vect_build_slp_tree): Remove inside_cost parameter; add stmt_cost_vec
	parameter; add stmt_cost_vec parameter to vect_get_and_check_slp_defs,
	vect_model_load_cost, and recursive vect_build_slp_tree calls; prevent
	calculating cost more than once for loads; remove inside_cost
	parameter from recursive vect_build_slp_tree calls; call
	record_stmt_cost instead of old calculation.
	(vect_analyze_slp_instance): Allocate stmt_cost_vec and save it with
	the instance; free it on premature exit; remove inside_cost parameter
	from vect_build_slp_tree call; add stmt_cost_vec parameter to
	vect_build_slp_tree call; don't set SLP_INSTANCE_INSIDE_OF_LOOP_COST.
	(new_bb_vec_info): Call init_cost.
	(destroy_bb_vec_info): Call destroy_cost_data.
	(vect_bb_vectorization_profitable_p): Call add_stmt_cost for each
	statement recorded with an SLP instance; call finish_cost instead of
	the old calculation.
	(vect_update_slp_costs_according_to_vf): Record statement costs from
	SLP instances, multiplying by the appropriate number of copies; don't
	update SLP_INSTANCE_INSIDE_OF_LOOP_COST.

From-SVN: r189460

1 files changed, 23 insertions, 31 deletions

diff --git a/gcc/tree-vect-loop.c b/gcc/tree-vect-loop.c
index b20c586..4ddc9fb 100644
--- a/gcc/tree-vect-loop.c
+++ b/gcc/tree-vect-loop.c
@@ -852,6 +852,7 @@ new_loop_vec_info (struct loop *loop)
   LOOP_VINFO_SLP_INSTANCES (res) = VEC_alloc (slp_instance, heap, 10);
   LOOP_VINFO_SLP_UNROLLING_FACTOR (res) = 1;
   LOOP_VINFO_PEELING_HTAB (res) = NULL;
+  LOOP_VINFO_TARGET_COST_DATA (res) = init_cost (loop);
   LOOP_VINFO_PEELING_FOR_GAPS (res) = false;
 
   return res;
@@ -929,6 +930,8 @@ destroy_loop_vec_info (loop_vec_info loop_vinfo, bool clean_stmts)
   if (LOOP_VINFO_PEELING_HTAB (loop_vinfo))
     htab_delete (LOOP_VINFO_PEELING_HTAB (loop_vinfo));
 
+  destroy_cost_data (LOOP_VINFO_TARGET_COST_DATA (loop_vinfo));
+
   free (loop_vinfo);
   loop->aux = NULL;
 }
@@ -1362,7 +1365,7 @@ vect_analyze_loop_operations (loop_vec_info loop_vinfo, bool slp)
                            "not vectorized: relevant phi not supported: ");
                   print_gimple_stmt (vect_dump, phi, 0, TDF_SLIM);
                 }
-              return false;
+	      return false;
             }
         }
 
@@ -2498,7 +2501,6 @@ vect_estimate_min_profitable_iters (loop_vec_info loop_vinfo)
   int nbbs = loop->num_nodes;
   int npeel = LOOP_PEELING_FOR_ALIGNMENT (loop_vinfo);
   int peel_guard_costs = 0;
-  int innerloop_iters = 0, factor;
   VEC (slp_instance, heap) *slp_instances;
   slp_instance instance;
 
@@ -2544,20 +2546,11 @@ vect_estimate_min_profitable_iters (loop_vec_info loop_vinfo)
      TODO: Consider assigning different costs to different scalar
      statements.  */
 
-  /* FORNOW.  */
-  if (loop->inner)
-    innerloop_iters = 50; /* FIXME */
-
   for (i = 0; i < nbbs; i++)
     {
       gimple_stmt_iterator si;
       basic_block bb = bbs[i];
 
-      if (bb->loop_father == loop->inner)
- 	factor = innerloop_iters;
-      else
- 	factor = 1;
-
       for (si = gsi_start_bb (bb); !gsi_end_p (si); gsi_next (&si))
 	{
 	  gimple stmt = gsi_stmt (si);
@@ -2575,7 +2568,6 @@ vect_estimate_min_profitable_iters (loop_vec_info loop_vinfo)
                  || !VECTORIZABLE_CYCLE_DEF (STMT_VINFO_DEF_TYPE (stmt_info))))
 	    continue;
 
-	  vec_inside_cost += STMT_VINFO_INSIDE_OF_LOOP_COST (stmt_info) * factor;
 	  /* FIXME: for stmts in the inner-loop in outer-loop vectorization,
 	     some of the "outside" costs are generated inside the outer-loop.  */
 	  vec_outside_cost += STMT_VINFO_OUTSIDE_OF_LOOP_COST (stmt_info);
@@ -2592,14 +2584,9 @@ vect_estimate_min_profitable_iters (loop_vec_info loop_vinfo)
 		    = vinfo_for_stmt (pattern_def_stmt);
                   if (STMT_VINFO_RELEVANT_P (pattern_def_stmt_info)
                       || STMT_VINFO_LIVE_P (pattern_def_stmt_info))
-		    {
-                      vec_inside_cost
-			+= STMT_VINFO_INSIDE_OF_LOOP_COST
-			   (pattern_def_stmt_info) * factor;
-                      vec_outside_cost
-			+= STMT_VINFO_OUTSIDE_OF_LOOP_COST
-			   (pattern_def_stmt_info);
-                    }
+		    vec_outside_cost
+		      += STMT_VINFO_OUTSIDE_OF_LOOP_COST
+		        (pattern_def_stmt_info);
 		}
 	    }
 	}
@@ -2725,11 +2712,12 @@ vect_estimate_min_profitable_iters (loop_vec_info loop_vinfo)
   /* Add SLP costs.  */
   slp_instances = LOOP_VINFO_SLP_INSTANCES (loop_vinfo);
   FOR_EACH_VEC_ELT (slp_instance, slp_instances, i, instance)
-    {
-      vec_outside_cost += SLP_INSTANCE_OUTSIDE_OF_LOOP_COST (instance);
-      vec_inside_cost += SLP_INSTANCE_INSIDE_OF_LOOP_COST (instance);
-    }
+    vec_outside_cost += SLP_INSTANCE_OUTSIDE_OF_LOOP_COST (instance);
 
+  /* Complete the target-specific cost calculation for the inside-of-loop
+     costs.  */
+  vec_inside_cost = finish_cost (LOOP_VINFO_TARGET_COST_DATA (loop_vinfo));
+  
   /* Calculate number of iterations required to make the vector version
      profitable, relative to the loop bodies only.  The following condition
      must hold true:
@@ -2826,10 +2814,10 @@ vect_model_reduction_cost (stmt_vec_info stmt_info, enum tree_code reduc_code,
   loop_vec_info loop_vinfo = STMT_VINFO_LOOP_VINFO (stmt_info);
   struct loop *loop = LOOP_VINFO_LOOP (loop_vinfo);
 
-
   /* Cost of reduction op inside loop.  */
-  STMT_VINFO_INSIDE_OF_LOOP_COST (stmt_info) 
-    += ncopies * vect_get_stmt_cost (vector_stmt);
+  unsigned inside_cost
+    = add_stmt_cost (LOOP_VINFO_TARGET_COST_DATA (loop_vinfo),
+		     ncopies, vector_stmt, stmt_info, 0);
 
   stmt = STMT_VINFO_STMT (stmt_info);
 
@@ -2915,7 +2903,7 @@ vect_model_reduction_cost (stmt_vec_info stmt_info, enum tree_code reduc_code,
 
   if (vect_print_dump_info (REPORT_COST))
     fprintf (vect_dump, "vect_model_reduction_cost: inside_cost = %d, "
-             "outside_cost = %d .", STMT_VINFO_INSIDE_OF_LOOP_COST (stmt_info),
+             "outside_cost = %d .", inside_cost,
              STMT_VINFO_OUTSIDE_OF_LOOP_COST (stmt_info));
 
   return true;
@@ -2929,16 +2917,20 @@ vect_model_reduction_cost (stmt_vec_info stmt_info, enum tree_code reduc_code,
 static void
 vect_model_induction_cost (stmt_vec_info stmt_info, int ncopies)
 {
+  loop_vec_info loop_vinfo = STMT_VINFO_LOOP_VINFO (stmt_info);
+
   /* loop cost for vec_loop.  */
-  STMT_VINFO_INSIDE_OF_LOOP_COST (stmt_info) 
-    = ncopies * vect_get_stmt_cost (vector_stmt);
+  unsigned inside_cost
+    = add_stmt_cost (LOOP_VINFO_TARGET_COST_DATA (loop_vinfo), ncopies,
+		     vector_stmt, stmt_info, 0);
+
   /* prologue cost for vec_init and vec_step.  */
   STMT_VINFO_OUTSIDE_OF_LOOP_COST (stmt_info)  
     = 2 * vect_get_stmt_cost (scalar_to_vec);
 
   if (vect_print_dump_info (REPORT_COST))
     fprintf (vect_dump, "vect_model_induction_cost: inside_cost = %d, "
-             "outside_cost = %d .", STMT_VINFO_INSIDE_OF_LOOP_COST (stmt_info),
+             "outside_cost = %d .", inside_cost,
              STMT_VINFO_OUTSIDE_OF_LOOP_COST (stmt_info));
 }