asan.c (create_cond_insert_point): Maintain profile.

	* asan.c (create_cond_insert_point): Maintain profile.
	* ipa-utils.c (ipa_merge_profiles): Be sure only IPA profiles are
	merged.
	* basic-block.h (struct basic_block_def): Remove frequency.
	(EDGE_FREQUENCY): Use to_frequency
	* bb-reorder.c (push_to_next_round_p): Use only IPA counts for global
	heuristics.
	(find_traces): Update to use to_frequency.
	(find_traces_1_round): Likewise; use only IPA counts.
	(bb_to_key): Likewise.
	(connect_traces): Use IPA counts only.
	(copy_bb_p): Update to use to_frequency.
	(fix_up_crossing_landing_pad): Likewise.
	(sanitize_hot_paths): Likewise.
	* bt-load.c (basic_block_freq): Likewise.
	* cfg.c (init_flow): Set count_max to uninitialized.
	(check_bb_profile): Remove frequencies; check counts.
	(dump_bb_info): Do not dump frequencies.
	(update_bb_profile_for_threading): Update counts only.
	(scale_bbs_frequencies_int): Likewise.
	(MAX_SAFE_MULTIPLIER): Remove.
	(scale_bbs_frequencies_gcov_type): Update counts only.
	(scale_bbs_frequencies_profile_count): Update counts only.
	(scale_bbs_frequencies): Update counts only.
	* cfg.h (struct control_flow_graph): Add count-max.
	(update_bb_profile_for_threading): Update prototype.
	* cfgbuild.c (find_bb_boundaries): Do not update frequencies.
	(find_many_sub_basic_blocks): Likewise.
	* cfgcleanup.c (try_forward_edges): Likewise.
	(try_crossjump_to_edge): Likewise.
	* cfgexpand.c (expand_gimple_cond): Likewise.
	(expand_gimple_tailcall): Likewise.
	(construct_init_block): Likewise.
	(construct_exit_block): Likewise.
	* cfghooks.c (verify_flow_info): Check consistency of counts.
	(dump_bb_for_graph): Do not dump frequencies.
	(split_block_1): Do not update frequencies.
	(split_edge): Do not update frequencies.
	(make_forwarder_block): Do not update frequencies.
	(duplicate_block): Do not update frequencies.
	(account_profile_record): Do not update frequencies.
	* cfgloop.c (find_subloop_latch_edge_by_profile): Use IPA counts
	for global heuristics.
	* cfgloopanal.c (average_num_loop_insns): Update to use to_frequency.
	(expected_loop_iterations_unbounded): Use counts only.
	* cfgloopmanip.c (scale_loop_profile): Simplify.
	(create_empty_loop_on_edge): Simplify
	(loopify): Simplify
	(duplicate_loop_to_header_edge): Simplify
	* cfgrtl.c (force_nonfallthru_and_redirect): Update profile.
	(update_br_prob_note): Take care of removing note when profile
	becomes undefined.
	(relink_block_chain): Do not dump frequency.
	(rtl_account_profile_record): Use to_frequency.
	* cgraph.c (symbol_table::create_edge): Convert count to ipa count.
	(cgraph_edge::redirect_call_stmt_to_calle): Conver tcount to ipa count.
	(cgraph_update_edges_for_call_stmt_node): Likewise.
	(cgraph_edge::verify_count_and_frequency): Update.
	(cgraph_node::verify_node): Temporarily disable frequency verification.
	* cgraphbuild.c (compute_call_stmt_bb_frequency): Use
	to_cgraph_frequency.
	(cgraph_edge::rebuild_edges): Convert to ipa counts.
	* cgraphunit.c (init_lowered_empty_function): Do not initialize
	frequencies.
	(cgraph_node::expand_thunk): Update profile.
	* except.c (dw2_build_landing_pads): Do not update frequency.
	* final.c (compute_alignments): Use to_frequency.
	(dump_basic_block_info): Do not dump frequency.
	* gimple-pretty-print.c (dump_profile): Do not dump frequency.
	(dump_gimple_bb_header): Do not dump frequency.
	* gimple-ssa-isolate-paths.c (isolate_path): Do not update frequency;
	do update count.
	* gimple-streamer-in.c (input_bb): Do not stream frequency.
	* gimple-streamer-out.c (output_bb): Do not stream frequency.
	* haifa-sched.c (sched_pressure_start_bb): Use to_freuqency.
	(init_before_recovery): Do not update frequency.
	(sched_create_recovery_edges): Do not update frequency.
	* hsa-gen.c (convert_switch_statements): Do not update frequency.
	* ipa-cp.c (ipcp_propagate_stage): Update search for max_count.
	(ipa_cp_c_finalize): Set max_count to uninitialized.
	* ipa-fnsummary.c (get_minimal_bb): Use counts.
	(param_change_prob): Use counts.
	* ipa-profile.c (ipa_profile_generate_summary): Do not summarize
	local profiles.
	* ipa-split.c (consider_split): Use to_frequency.
	(split_function): Use to_frequency.
	* ira-build.c (loop_compare_func): Likewise.
	(mark_loops_for_removal): Likewise.
	(mark_all_loops_for_removal): Likewise.
	* loop-doloop.c (doloop_modify): Do not update frequency.
	* loop-unroll.c (unroll_loop_runtime_iterations): Do not update
	frequency.
	* lto-streamer-in.c (input_function): Update count_max.
	* omp-expand.c (expand_omp_taskreg): Update count_max.
	* omp-simd-clone.c (simd_clone_adjust): Update profile.
	* predict.c (maybe_hot_frequency_p): Use to_frequency.
	(maybe_hot_count_p): Use ipa counts only.
	(maybe_hot_bb_p): Simplify.
	(maybe_hot_edge_p): Simplify.
	(probably_never_executed): Do not take frequency argument.
	(probably_never_executed_bb_p): Do not pass frequency.
	(probably_never_executed_edge_p): Likewise.
	(combine_predictions_for_bb): Check that profile is nonzero.
	(propagate_freq): Do not set frequency.
	(drop_profile): Simplify.
	(counts_to_freqs): Simplify.
	(expensive_function_p): Use to_frequency.
	(propagate_unlikely_bbs_forward): Simplify.
	(determine_unlikely_bbs): Simplify.
	(estimate_bb_frequencies): Add hack to silence graphite issues.
	(compute_function_frequency): Use ipa counts.
	(pass_profile::execute): Update.
	(rebuild_frequencies): Use counts only.
	(force_edge_cold): Use counts only.
	* profile-count.c (profile_count::dump): Dump new count types.
	(profile_count::differs_from_p): Check compatiblity.
	(profile_count::to_frequency): New function.
	(profile_count::to_cgraph_frequency): New function.
	* profile-count.h (struct function): Declare.
	(enum profile_quality): Add profile_guessed_local and
	profile_guessed_global0.
	(class profile_proability): Decrease number of bits to 29;
	update from_reg_br_prob_note and to_reg_br_prob_note.
	(class profile_count: Update comment; decrease number of bits
	to 61. Check compatibility.
	(profile_count::compatible_p): New private member function.
	(profile_count::ipa_p): New member function.
	(profile_count::operator<): Handle global zero correctly.
	(profile_count::operator>): Handle global zero correctly.
	(profile_count::operator<=): Handle global zero correctly.
	(profile_count::operator>=): Handle global zero correctly.
	(profile_count::nonzero_p): New member function.
	(profile_count::force_nonzero): New member function.
	(profile_count::max): New member function.
	(profile_count::apply_scale): Handle IPA scalling.
	(profile_count::guessed_local): New member function.
	(profile_count::global0): New member function.
	(profile_count::ipa): New member function.
	(profile_count::to_frequency): Declare.
	(profile_count::to_cgraph_frequency): Declare.
	* profile.c (OVERLAP_BASE): Delete.
	(compute_frequency_overlap): Delete.
	(compute_branch_probabilities): Do not use compute_frequency_overlap.
	* regs.h (REG_FREQ_FROM_BB): Use to_frequency.
	* sched-ebb.c (rank): Use counts only.
	* shrink-wrap.c (handle_simple_exit): Use counts only.
	(try_shrink_wrapping): Use counts only.
	(place_prologue_for_one_component): Use counts only.
	* tracer.c (find_best_predecessor): Use to_frequency.
	(find_trace): Use to_frequency.
	(tail_duplicate): Use to_frequency.
	* trans-mem.c (expand_transaction): Do not update frequency.
	* tree-call-cdce.c: Do not update frequency. 
	* tree-cfg.c (gimple_find_sub_bbs): Likewise.
	(gimple_merge_blocks): Likewise.
	(gimple_split_edge): Likewise.
	(gimple_duplicate_sese_region): Likewise.
	(gimple_duplicate_sese_tail): Likewise.
	(move_sese_region_to_fn): Likewise.
	(gimple_account_profile_record): Likewise.
	(insert_cond_bb): Likewise.
	* tree-complex.c (expand_complex_div_wide): Likewise.
	* tree-eh.c (lower_resx): Update profile.
	* tree-inline.c (copy_bb): Simplify count scaling; do not scale
	frequencies.
	(initialize_cfun): Do not initialize frequencies
	(freqs_to_counts): Delete.
	(copy_cfg_body): Ignore count parameter.
	(copy_body): Update.
	(expand_call_inline): Update count_max.
	(optimize_inline_calls): Update count_max.
	(tree_function_versioning): Update count_max.
	* tree-ssa-coalesce.c (coalesce_cost_bb): Use to_frequency.
	* tree-ssa-ifcombine.c (update_profile_after_ifcombine): Do not update
	frequency.
	* tree-ssa-loop-im.c (execute_sm_if_changed): Use counts only.
	* tree-ssa-loop-ivcanon.c (unloop_loops): Do not update freuqency.
	(try_peel_loop): Likewise.
	* tree-ssa-loop-ivopts.c (get_scaled_computation_cost_at): Use
	to_frequency.
	* tree-ssa-loop-manip.c (niter_for_unrolled_loop): Pass -1.
	(tree_transform_and_unroll_loop): Do not use frequencies
	* tree-ssa-loop-niter.c (estimate_numbers_of_iterations):
	Use reliable prediction only.
	* tree-ssa-loop-unswitch.c (hoist_guard): Do not use frequencies.
	* tree-ssa-sink.c (select_best_block): Use to_frequency.
	* tree-ssa-tail-merge.c (replace_block_by): Temporarily disable
	probability scaling.
	* tree-ssa-threadupdate.c (create_block_for_threading): Do
	not update frequency
	(any_remaining_duplicated_blocks): Likewise.
	(update_profile): Likewise.
	(estimated_freqs_path): Delete.
	(freqs_to_counts_path): Delete.
	(clear_counts_path): Delete.
	(ssa_fix_duplicate_block_edges): Likewise.
	(duplicate_thread_path): Likewise.
	* tree-switch-conversion.c (gen_inbound_check): Use counts.
	* tree-tailcall.c (decrease_profile): Do not update frequency.
	(eliminate_tail_call): Likewise.
	* tree-vect-loop-manip.c (vect_do_peeling): Likewise.
	* tree-vect-loop.c (scale_profile_for_vect_loop): Likewise.
	(optimize_mask_stores): Likewise.
	* tree-vect-stmts.c (vectorizable_simd_clone_call): Likewise.
	* ubsan.c (ubsan_expand_null_ifn): Update profile.
	(ubsan_expand_ptr_ifn): Update profile.
	* value-prof.c (gimple_ic): Simplify.
	* value-prof.h (gimple_ic): Update prototype.
	* ipa-inline-transform.c (inline_transform): Fix scaling conditoins.
	* ipa-inline.c (compute_uninlined_call_time): Be sure that
	counts are nonzero.
	(want_inline_self_recursive_call_p): Likewise.
	(resolve_noninline_speculation): Only cummulate defined counts.
	(inline_small_functions): Use nonzero_p.
	(ipa_inline): Do not access freed node.

Unknown ChangeLog:

2017-11-02  Jan Hubicka  <hubicka@ucw.cz>

	* testsuite/gcc.dg/no-strict-overflow-3.c (foo): Update magic
	value to not clash with frequency.
	* testsuite/gcc.dg/strict-overflow-3.c (foo): Likewise.
	* testsuite/gcc.dg/tree-ssa/builtin-sprintf-2.c: Update template.
	* testsuite/gcc.dg/tree-ssa/dump-2.c: Update template.
	* testsuite/gcc.dg/tree-ssa/ifc-10.c: Update template.
	* testsuite/gcc.dg/tree-ssa/ifc-11.c: Update template.
	* testsuite/gcc.dg/tree-ssa/ifc-12.c: Update template.
	* testsuite/gcc.dg/tree-ssa/ifc-20040816-1.c: Update template.
	* testsuite/gcc.dg/tree-ssa/ifc-20040816-2.c: Update template.
	* testsuite/gcc.dg/tree-ssa/ifc-5.c: Update template.
	* testsuite/gcc.dg/tree-ssa/ifc-8.c: Update template.
	* testsuite/gcc.dg/tree-ssa/ifc-9.c: Update template.
	* testsuite/gcc.dg/tree-ssa/ifc-cd.c: Update template.
	* testsuite/gcc.dg/tree-ssa/ifc-pr56541.c: Update template.
	* testsuite/gcc.dg/tree-ssa/ifc-pr68583.c: Update template.
	* testsuite/gcc.dg/tree-ssa/ifc-pr69489-1.c: Update template.
	* testsuite/gcc.dg/tree-ssa/ifc-pr69489-2.c: Update template.
	* testsuite/gcc.target/i386/pr61403.c: Update template.

From-SVN: r254379

1 files changed, 31 insertions, 82 deletions

diff --git a/gcc/tree-inline.c b/gcc/tree-inline.c
index 597addb..040799c 100644
--- a/gcc/tree-inline.c
+++ b/gcc/tree-inline.c
@@ -1763,16 +1763,15 @@ remap_gimple_stmt (gimple *stmt, copy_body_data *id)
    later  */
 
 static basic_block
-copy_bb (copy_body_data *id, basic_block bb, int frequency_scale,
+copy_bb (copy_body_data *id, basic_block bb,
          profile_count num, profile_count den)
 {
   gimple_stmt_iterator gsi, copy_gsi, seq_gsi;
   basic_block copy_basic_block;
   tree decl;
-  gcov_type freq;
   basic_block prev;
-  bool scale = num.initialized_p ()
-	       && (den > 0 || num == profile_count::zero ());
+  bool scale = !num.initialized_p ()
+	       || (den.nonzero_p () || num == profile_count::zero ());
 
   /* Search for previous copied basic block.  */
   prev = bb->prev_bb;
@@ -1784,15 +1783,8 @@ copy_bb (copy_body_data *id, basic_block bb, int frequency_scale,
   copy_basic_block = create_basic_block (NULL, (basic_block) prev->aux);
   if (scale)
     copy_basic_block->count = bb->count.apply_scale (num, den);
-
-  /* We are going to rebuild frequencies from scratch.  These values
-     have just small importance to drive canonicalize_loop_headers.  */
-  freq = apply_scale ((gcov_type)bb->frequency, frequency_scale);
-
-  /* We recompute frequencies after inlining, so this is quite safe.  */
-  if (freq > BB_FREQ_MAX)
-    freq = BB_FREQ_MAX;
-  copy_basic_block->frequency = freq;
+  else if (num.initialized_p ())
+    copy_basic_block->count = bb->count;
 
   copy_gsi = gsi_start_bb (copy_basic_block);
 
@@ -2068,8 +2060,8 @@ copy_bb (copy_body_data *id, basic_block bb, int frequency_scale,
 			      fprintf (dump_file,
 				       "Orig bb: %i, orig bb freq %i, new bb freq %i\n",
 				       bb->index,
-				       bb->frequency,
-				       copy_basic_block->frequency);
+				       bb->count.to_frequency (cfun),
+				       copy_basic_block->count.to_frequency (cfun));
 			    }
 			}
 		    }
@@ -2507,11 +2499,8 @@ initialize_cfun (tree new_fndecl, tree callee_fndecl, profile_count count)
 
   profile_status_for_fn (cfun) = profile_status_for_fn (src_cfun);
 
-  /* FIXME: When all counts are known to be zero, scaling is also meaningful.
-   */
   if (ENTRY_BLOCK_PTR_FOR_FN (src_cfun)->count.initialized_p ()
-      && count.initialized_p ()
-      && ENTRY_BLOCK_PTR_FOR_FN (src_cfun)->count.initialized_p ())
+      && count.ipa ().initialized_p ())
     {
       ENTRY_BLOCK_PTR_FOR_FN (cfun)->count =
 	ENTRY_BLOCK_PTR_FOR_FN (src_cfun)->count.apply_scale (count,
@@ -2520,10 +2509,13 @@ initialize_cfun (tree new_fndecl, tree callee_fndecl, profile_count count)
 	EXIT_BLOCK_PTR_FOR_FN (src_cfun)->count.apply_scale (count,
 				    ENTRY_BLOCK_PTR_FOR_FN (src_cfun)->count);
     }
-  ENTRY_BLOCK_PTR_FOR_FN (cfun)->frequency
-    = ENTRY_BLOCK_PTR_FOR_FN (src_cfun)->frequency;
-  EXIT_BLOCK_PTR_FOR_FN (cfun)->frequency =
-    EXIT_BLOCK_PTR_FOR_FN (src_cfun)->frequency;
+  else
+    {
+      ENTRY_BLOCK_PTR_FOR_FN (cfun)->count
+	   = ENTRY_BLOCK_PTR_FOR_FN (src_cfun)->count;
+      EXIT_BLOCK_PTR_FOR_FN (cfun)->count
+	   = EXIT_BLOCK_PTR_FOR_FN (src_cfun)->count;
+    }
   if (src_cfun->eh)
     init_eh_for_function ();
 
@@ -2680,27 +2672,11 @@ redirect_all_calls (copy_body_data * id, basic_block bb)
     }
 }
 
-/* Convert estimated frequencies into counts for NODE, scaling COUNT
-   with each bb's frequency. Used when NODE has a 0-weight entry
-   but we are about to inline it into a non-zero count call bb.
-   See the comments for handle_missing_profiles() in predict.c for
-   when this can happen for COMDATs.  */
-
-void
-freqs_to_counts (struct cgraph_node *node, profile_count count)
-{
-  basic_block bb;
-  struct function *fn = DECL_STRUCT_FUNCTION (node->decl);
-
-  FOR_ALL_BB_FN(bb, fn)
-    bb->count = count.apply_scale (bb->frequency, BB_FREQ_MAX);
-}
-
 /* Make a copy of the body of FN so that it can be inserted inline in
    another function.  Walks FN via CFG, returns new fndecl.  */
 
 static tree
-copy_cfg_body (copy_body_data * id, profile_count count, int frequency_scale,
+copy_cfg_body (copy_body_data * id, profile_count,
 	       basic_block entry_block_map, basic_block exit_block_map,
 	       basic_block new_entry)
 {
@@ -2712,31 +2688,10 @@ copy_cfg_body (copy_body_data * id, profile_count count, int frequency_scale,
   tree new_fndecl = NULL;
   bool need_debug_cleanup = false;
   int last;
-  int incoming_frequency = 0;
-  profile_count incoming_count = profile_count::zero ();
-  profile_count num = count;
   profile_count den = ENTRY_BLOCK_PTR_FOR_FN (src_cfun)->count;
-  bool scale = num.initialized_p ()
-	       && (den > 0 || num == profile_count::zero ());
-
-  /* This can happen for COMDAT routines that end up with 0 counts
-     despite being called (see the comments for handle_missing_profiles()
-     in predict.c as to why). Apply counts to the blocks in the callee
-     before inlining, using the guessed edge frequencies, so that we don't
-     end up with a 0-count inline body which can confuse downstream
-     optimizations such as function splitting.  */
-  if (!(ENTRY_BLOCK_PTR_FOR_FN (src_cfun)->count > 0) && count > 0)
-    {
-      /* Apply the larger of the call bb count and the total incoming
-         call edge count to the callee.  */
-      profile_count in_count = profile_count::zero ();
-      struct cgraph_edge *in_edge;
-      for (in_edge = id->src_node->callers; in_edge;
-           in_edge = in_edge->next_caller)
-	if (in_edge->count.initialized_p ())
-          in_count += in_edge->count;
-      freqs_to_counts (id->src_node, count > in_count ? count : in_count);
-    }
+  profile_count num = entry_block_map->count;
+
+  cfun_to_copy = id->src_cfun = DECL_STRUCT_FUNCTION (callee_fndecl);
 
   /* Register specific tree functions.  */
   gimple_register_cfg_hooks ();
@@ -2750,25 +2705,18 @@ copy_cfg_body (copy_body_data * id, profile_count count, int frequency_scale,
     {
       edge e;
       edge_iterator ei;
+      den = profile_count::zero ();
 
       FOR_EACH_EDGE (e, ei, new_entry->preds)
 	if (!e->src->aux)
-	  incoming_frequency += EDGE_FREQUENCY (e);
-      if (scale)
-        incoming_count = incoming_count.apply_scale (num, den);
-      else
-	incoming_count = profile_count::uninitialized ();
-      incoming_frequency
-	= apply_scale ((gcov_type)incoming_frequency, frequency_scale);
-      ENTRY_BLOCK_PTR_FOR_FN (cfun)->count = incoming_count;
-      ENTRY_BLOCK_PTR_FOR_FN (cfun)->frequency = incoming_frequency;
+	  den += e->count ();
+      ENTRY_BLOCK_PTR_FOR_FN (cfun)->count = den;
     }
 
   /* Must have a CFG here at this point.  */
   gcc_assert (ENTRY_BLOCK_PTR_FOR_FN
 	      (DECL_STRUCT_FUNCTION (callee_fndecl)));
 
-  cfun_to_copy = id->src_cfun = DECL_STRUCT_FUNCTION (callee_fndecl);
 
   ENTRY_BLOCK_PTR_FOR_FN (cfun_to_copy)->aux = entry_block_map;
   EXIT_BLOCK_PTR_FOR_FN (cfun_to_copy)->aux = exit_block_map;
@@ -2784,7 +2732,7 @@ copy_cfg_body (copy_body_data * id, profile_count count, int frequency_scale,
   FOR_EACH_BB_FN (bb, cfun_to_copy)
     if (!id->blocks_to_copy || bitmap_bit_p (id->blocks_to_copy, bb->index))
       {
-	basic_block new_bb = copy_bb (id, bb, frequency_scale, num, den);
+	basic_block new_bb = copy_bb (id, bb, num, den);
 	bb->aux = new_bb;
 	new_bb->aux = bb;
 	new_bb->loop_father = entry_block_map->loop_father;
@@ -3011,7 +2959,7 @@ copy_tree_body (copy_body_data *id)
    another function.  */
 
 static tree
-copy_body (copy_body_data *id, profile_count count, int frequency_scale,
+copy_body (copy_body_data *id, profile_count count,
 	   basic_block entry_block_map, basic_block exit_block_map,
 	   basic_block new_entry)
 {
@@ -3020,7 +2968,7 @@ copy_body (copy_body_data *id, profile_count count, int frequency_scale,
 
   /* If this body has a CFG, walk CFG and copy.  */
   gcc_assert (ENTRY_BLOCK_PTR_FOR_FN (DECL_STRUCT_FUNCTION (fndecl)));
-  body = copy_cfg_body (id, count, frequency_scale, entry_block_map, exit_block_map,
+  body = copy_cfg_body (id, count, entry_block_map, exit_block_map,
 			new_entry);
   copy_debug_stmts (id);
 
@@ -4771,7 +4719,6 @@ expand_call_inline (basic_block bb, gimple *stmt, copy_body_data *id)
      a self-referential call; if we're calling ourselves, we need to
      duplicate our body before altering anything.  */
   copy_body (id, cg_edge->callee->count,
-  	     GCOV_COMPUTE_SCALE (cg_edge->frequency, CGRAPH_FREQ_BASE),
 	     bb, return_block, NULL);
 
   reset_debug_bindings (id, stmt_gsi);
@@ -5146,6 +5093,7 @@ optimize_inline_calls (tree fn)
     }
 
   /* Fold queued statements.  */
+  counts_to_freqs ();
   fold_marked_statements (last, id.statements_to_fold);
   delete id.statements_to_fold;
 
@@ -6090,7 +6038,7 @@ tree_function_versioning (tree old_decl, tree new_decl,
     }
 
   /* Copy the Function's body.  */
-  copy_body (&id, old_entry_block->count, REG_BR_PROB_BASE,
+  copy_body (&id, old_entry_block->count,
 	     ENTRY_BLOCK_PTR_FOR_FN (cfun), EXIT_BLOCK_PTR_FOR_FN (cfun),
 	     new_entry);
 
@@ -6122,6 +6070,7 @@ tree_function_versioning (tree old_decl, tree new_decl,
   free_dominance_info (CDI_DOMINATORS);
   free_dominance_info (CDI_POST_DOMINATORS);
 
+  counts_to_freqs ();
   fold_marked_statements (0, id.statements_to_fold);
   delete id.statements_to_fold;
   delete_unreachable_blocks_update_callgraph (&id);
@@ -6141,20 +6090,20 @@ tree_function_versioning (tree old_decl, tree new_decl,
       struct cgraph_edge *e;
       rebuild_frequencies ();
 
-      new_version_node->count = ENTRY_BLOCK_PTR_FOR_FN (cfun)->count;
+      new_version_node->count = ENTRY_BLOCK_PTR_FOR_FN (cfun)->count.ipa ();
       for (e = new_version_node->callees; e; e = e->next_callee)
 	{
 	  basic_block bb = gimple_bb (e->call_stmt);
 	  e->frequency = compute_call_stmt_bb_frequency (current_function_decl,
 							 bb);
-	  e->count = bb->count;
+	  e->count = bb->count.ipa ();
 	}
       for (e = new_version_node->indirect_calls; e; e = e->next_callee)
 	{
 	  basic_block bb = gimple_bb (e->call_stmt);
 	  e->frequency = compute_call_stmt_bb_frequency (current_function_decl,
 							 bb);
-	  e->count = bb->count;
+	  e->count = bb->count.ipa ();
 	}
     }