asan.c (create_cond_insert_point): Maintain profile.

	* asan.c (create_cond_insert_point): Maintain profile.
	* ipa-utils.c (ipa_merge_profiles): Be sure only IPA profiles are
	merged.
	* basic-block.h (struct basic_block_def): Remove frequency.
	(EDGE_FREQUENCY): Use to_frequency
	* bb-reorder.c (push_to_next_round_p): Use only IPA counts for global
	heuristics.
	(find_traces): Update to use to_frequency.
	(find_traces_1_round): Likewise; use only IPA counts.
	(bb_to_key): Likewise.
	(connect_traces): Use IPA counts only.
	(copy_bb_p): Update to use to_frequency.
	(fix_up_crossing_landing_pad): Likewise.
	(sanitize_hot_paths): Likewise.
	* bt-load.c (basic_block_freq): Likewise.
	* cfg.c (init_flow): Set count_max to uninitialized.
	(check_bb_profile): Remove frequencies; check counts.
	(dump_bb_info): Do not dump frequencies.
	(update_bb_profile_for_threading): Update counts only.
	(scale_bbs_frequencies_int): Likewise.
	(MAX_SAFE_MULTIPLIER): Remove.
	(scale_bbs_frequencies_gcov_type): Update counts only.
	(scale_bbs_frequencies_profile_count): Update counts only.
	(scale_bbs_frequencies): Update counts only.
	* cfg.h (struct control_flow_graph): Add count-max.
	(update_bb_profile_for_threading): Update prototype.
	* cfgbuild.c (find_bb_boundaries): Do not update frequencies.
	(find_many_sub_basic_blocks): Likewise.
	* cfgcleanup.c (try_forward_edges): Likewise.
	(try_crossjump_to_edge): Likewise.
	* cfgexpand.c (expand_gimple_cond): Likewise.
	(expand_gimple_tailcall): Likewise.
	(construct_init_block): Likewise.
	(construct_exit_block): Likewise.
	* cfghooks.c (verify_flow_info): Check consistency of counts.
	(dump_bb_for_graph): Do not dump frequencies.
	(split_block_1): Do not update frequencies.
	(split_edge): Do not update frequencies.
	(make_forwarder_block): Do not update frequencies.
	(duplicate_block): Do not update frequencies.
	(account_profile_record): Do not update frequencies.
	* cfgloop.c (find_subloop_latch_edge_by_profile): Use IPA counts
	for global heuristics.
	* cfgloopanal.c (average_num_loop_insns): Update to use to_frequency.
	(expected_loop_iterations_unbounded): Use counts only.
	* cfgloopmanip.c (scale_loop_profile): Simplify.
	(create_empty_loop_on_edge): Simplify
	(loopify): Simplify
	(duplicate_loop_to_header_edge): Simplify
	* cfgrtl.c (force_nonfallthru_and_redirect): Update profile.
	(update_br_prob_note): Take care of removing note when profile
	becomes undefined.
	(relink_block_chain): Do not dump frequency.
	(rtl_account_profile_record): Use to_frequency.
	* cgraph.c (symbol_table::create_edge): Convert count to ipa count.
	(cgraph_edge::redirect_call_stmt_to_calle): Conver tcount to ipa count.
	(cgraph_update_edges_for_call_stmt_node): Likewise.
	(cgraph_edge::verify_count_and_frequency): Update.
	(cgraph_node::verify_node): Temporarily disable frequency verification.
	* cgraphbuild.c (compute_call_stmt_bb_frequency): Use
	to_cgraph_frequency.
	(cgraph_edge::rebuild_edges): Convert to ipa counts.
	* cgraphunit.c (init_lowered_empty_function): Do not initialize
	frequencies.
	(cgraph_node::expand_thunk): Update profile.
	* except.c (dw2_build_landing_pads): Do not update frequency.
	* final.c (compute_alignments): Use to_frequency.
	(dump_basic_block_info): Do not dump frequency.
	* gimple-pretty-print.c (dump_profile): Do not dump frequency.
	(dump_gimple_bb_header): Do not dump frequency.
	* gimple-ssa-isolate-paths.c (isolate_path): Do not update frequency;
	do update count.
	* gimple-streamer-in.c (input_bb): Do not stream frequency.
	* gimple-streamer-out.c (output_bb): Do not stream frequency.
	* haifa-sched.c (sched_pressure_start_bb): Use to_freuqency.
	(init_before_recovery): Do not update frequency.
	(sched_create_recovery_edges): Do not update frequency.
	* hsa-gen.c (convert_switch_statements): Do not update frequency.
	* ipa-cp.c (ipcp_propagate_stage): Update search for max_count.
	(ipa_cp_c_finalize): Set max_count to uninitialized.
	* ipa-fnsummary.c (get_minimal_bb): Use counts.
	(param_change_prob): Use counts.
	* ipa-profile.c (ipa_profile_generate_summary): Do not summarize
	local profiles.
	* ipa-split.c (consider_split): Use to_frequency.
	(split_function): Use to_frequency.
	* ira-build.c (loop_compare_func): Likewise.
	(mark_loops_for_removal): Likewise.
	(mark_all_loops_for_removal): Likewise.
	* loop-doloop.c (doloop_modify): Do not update frequency.
	* loop-unroll.c (unroll_loop_runtime_iterations): Do not update
	frequency.
	* lto-streamer-in.c (input_function): Update count_max.
	* omp-expand.c (expand_omp_taskreg): Update count_max.
	* omp-simd-clone.c (simd_clone_adjust): Update profile.
	* predict.c (maybe_hot_frequency_p): Use to_frequency.
	(maybe_hot_count_p): Use ipa counts only.
	(maybe_hot_bb_p): Simplify.
	(maybe_hot_edge_p): Simplify.
	(probably_never_executed): Do not take frequency argument.
	(probably_never_executed_bb_p): Do not pass frequency.
	(probably_never_executed_edge_p): Likewise.
	(combine_predictions_for_bb): Check that profile is nonzero.
	(propagate_freq): Do not set frequency.
	(drop_profile): Simplify.
	(counts_to_freqs): Simplify.
	(expensive_function_p): Use to_frequency.
	(propagate_unlikely_bbs_forward): Simplify.
	(determine_unlikely_bbs): Simplify.
	(estimate_bb_frequencies): Add hack to silence graphite issues.
	(compute_function_frequency): Use ipa counts.
	(pass_profile::execute): Update.
	(rebuild_frequencies): Use counts only.
	(force_edge_cold): Use counts only.
	* profile-count.c (profile_count::dump): Dump new count types.
	(profile_count::differs_from_p): Check compatiblity.
	(profile_count::to_frequency): New function.
	(profile_count::to_cgraph_frequency): New function.
	* profile-count.h (struct function): Declare.
	(enum profile_quality): Add profile_guessed_local and
	profile_guessed_global0.
	(class profile_proability): Decrease number of bits to 29;
	update from_reg_br_prob_note and to_reg_br_prob_note.
	(class profile_count: Update comment; decrease number of bits
	to 61. Check compatibility.
	(profile_count::compatible_p): New private member function.
	(profile_count::ipa_p): New member function.
	(profile_count::operator<): Handle global zero correctly.
	(profile_count::operator>): Handle global zero correctly.
	(profile_count::operator<=): Handle global zero correctly.
	(profile_count::operator>=): Handle global zero correctly.
	(profile_count::nonzero_p): New member function.
	(profile_count::force_nonzero): New member function.
	(profile_count::max): New member function.
	(profile_count::apply_scale): Handle IPA scalling.
	(profile_count::guessed_local): New member function.
	(profile_count::global0): New member function.
	(profile_count::ipa): New member function.
	(profile_count::to_frequency): Declare.
	(profile_count::to_cgraph_frequency): Declare.
	* profile.c (OVERLAP_BASE): Delete.
	(compute_frequency_overlap): Delete.
	(compute_branch_probabilities): Do not use compute_frequency_overlap.
	* regs.h (REG_FREQ_FROM_BB): Use to_frequency.
	* sched-ebb.c (rank): Use counts only.
	* shrink-wrap.c (handle_simple_exit): Use counts only.
	(try_shrink_wrapping): Use counts only.
	(place_prologue_for_one_component): Use counts only.
	* tracer.c (find_best_predecessor): Use to_frequency.
	(find_trace): Use to_frequency.
	(tail_duplicate): Use to_frequency.
	* trans-mem.c (expand_transaction): Do not update frequency.
	* tree-call-cdce.c: Do not update frequency. 
	* tree-cfg.c (gimple_find_sub_bbs): Likewise.
	(gimple_merge_blocks): Likewise.
	(gimple_split_edge): Likewise.
	(gimple_duplicate_sese_region): Likewise.
	(gimple_duplicate_sese_tail): Likewise.
	(move_sese_region_to_fn): Likewise.
	(gimple_account_profile_record): Likewise.
	(insert_cond_bb): Likewise.
	* tree-complex.c (expand_complex_div_wide): Likewise.
	* tree-eh.c (lower_resx): Update profile.
	* tree-inline.c (copy_bb): Simplify count scaling; do not scale
	frequencies.
	(initialize_cfun): Do not initialize frequencies
	(freqs_to_counts): Delete.
	(copy_cfg_body): Ignore count parameter.
	(copy_body): Update.
	(expand_call_inline): Update count_max.
	(optimize_inline_calls): Update count_max.
	(tree_function_versioning): Update count_max.
	* tree-ssa-coalesce.c (coalesce_cost_bb): Use to_frequency.
	* tree-ssa-ifcombine.c (update_profile_after_ifcombine): Do not update
	frequency.
	* tree-ssa-loop-im.c (execute_sm_if_changed): Use counts only.
	* tree-ssa-loop-ivcanon.c (unloop_loops): Do not update freuqency.
	(try_peel_loop): Likewise.
	* tree-ssa-loop-ivopts.c (get_scaled_computation_cost_at): Use
	to_frequency.
	* tree-ssa-loop-manip.c (niter_for_unrolled_loop): Pass -1.
	(tree_transform_and_unroll_loop): Do not use frequencies
	* tree-ssa-loop-niter.c (estimate_numbers_of_iterations):
	Use reliable prediction only.
	* tree-ssa-loop-unswitch.c (hoist_guard): Do not use frequencies.
	* tree-ssa-sink.c (select_best_block): Use to_frequency.
	* tree-ssa-tail-merge.c (replace_block_by): Temporarily disable
	probability scaling.
	* tree-ssa-threadupdate.c (create_block_for_threading): Do
	not update frequency
	(any_remaining_duplicated_blocks): Likewise.
	(update_profile): Likewise.
	(estimated_freqs_path): Delete.
	(freqs_to_counts_path): Delete.
	(clear_counts_path): Delete.
	(ssa_fix_duplicate_block_edges): Likewise.
	(duplicate_thread_path): Likewise.
	* tree-switch-conversion.c (gen_inbound_check): Use counts.
	* tree-tailcall.c (decrease_profile): Do not update frequency.
	(eliminate_tail_call): Likewise.
	* tree-vect-loop-manip.c (vect_do_peeling): Likewise.
	* tree-vect-loop.c (scale_profile_for_vect_loop): Likewise.
	(optimize_mask_stores): Likewise.
	* tree-vect-stmts.c (vectorizable_simd_clone_call): Likewise.
	* ubsan.c (ubsan_expand_null_ifn): Update profile.
	(ubsan_expand_ptr_ifn): Update profile.
	* value-prof.c (gimple_ic): Simplify.
	* value-prof.h (gimple_ic): Update prototype.
	* ipa-inline-transform.c (inline_transform): Fix scaling conditoins.
	* ipa-inline.c (compute_uninlined_call_time): Be sure that
	counts are nonzero.
	(want_inline_self_recursive_call_p): Likewise.
	(resolve_noninline_speculation): Only cummulate defined counts.
	(inline_small_functions): Use nonzero_p.
	(ipa_inline): Do not access freed node.

Unknown ChangeLog:

2017-11-02  Jan Hubicka  <hubicka@ucw.cz>

	* testsuite/gcc.dg/no-strict-overflow-3.c (foo): Update magic
	value to not clash with frequency.
	* testsuite/gcc.dg/strict-overflow-3.c (foo): Likewise.
	* testsuite/gcc.dg/tree-ssa/builtin-sprintf-2.c: Update template.
	* testsuite/gcc.dg/tree-ssa/dump-2.c: Update template.
	* testsuite/gcc.dg/tree-ssa/ifc-10.c: Update template.
	* testsuite/gcc.dg/tree-ssa/ifc-11.c: Update template.
	* testsuite/gcc.dg/tree-ssa/ifc-12.c: Update template.
	* testsuite/gcc.dg/tree-ssa/ifc-20040816-1.c: Update template.
	* testsuite/gcc.dg/tree-ssa/ifc-20040816-2.c: Update template.
	* testsuite/gcc.dg/tree-ssa/ifc-5.c: Update template.
	* testsuite/gcc.dg/tree-ssa/ifc-8.c: Update template.
	* testsuite/gcc.dg/tree-ssa/ifc-9.c: Update template.
	* testsuite/gcc.dg/tree-ssa/ifc-cd.c: Update template.
	* testsuite/gcc.dg/tree-ssa/ifc-pr56541.c: Update template.
	* testsuite/gcc.dg/tree-ssa/ifc-pr68583.c: Update template.
	* testsuite/gcc.dg/tree-ssa/ifc-pr69489-1.c: Update template.
	* testsuite/gcc.dg/tree-ssa/ifc-pr69489-2.c: Update template.
	* testsuite/gcc.target/i386/pr61403.c: Update template.

From-SVN: r254379

1 files changed, 81 insertions, 66 deletions

diff --git a/gcc/predict.c b/gcc/predict.c
index 0a85d0b..cf42ccb 100644
--- a/gcc/predict.c
+++ b/gcc/predict.c
@@ -137,12 +137,12 @@ maybe_hot_frequency_p (struct function *fun, int freq)
   if (profile_status_for_fn (fun) == PROFILE_ABSENT)
     return true;
   if (node->frequency == NODE_FREQUENCY_EXECUTED_ONCE
-      && freq < (ENTRY_BLOCK_PTR_FOR_FN (fun)->frequency * 2 / 3))
+      && freq < (ENTRY_BLOCK_PTR_FOR_FN (fun)->count.to_frequency (cfun) * 2 / 3))
     return false;
   if (PARAM_VALUE (HOT_BB_FREQUENCY_FRACTION) == 0)
     return false;
   if (freq * PARAM_VALUE (HOT_BB_FREQUENCY_FRACTION)
-      < ENTRY_BLOCK_PTR_FOR_FN (fun)->frequency)
+      < ENTRY_BLOCK_PTR_FOR_FN (fun)->count.to_frequency (cfun))
     return false;
   return true;
 }
@@ -175,10 +175,14 @@ set_hot_bb_threshold (gcov_type min)
 /* Return TRUE if frequency FREQ is considered to be hot.  */
 
 bool
-maybe_hot_count_p (struct function *, profile_count count)
+maybe_hot_count_p (struct function *fun, profile_count count)
 {
   if (!count.initialized_p ())
     return true;
+  if (!count.ipa_p ())
+    return maybe_hot_frequency_p (fun, count.to_frequency (fun));
+  if (count.ipa () == profile_count::zero ())
+    return false;
   /* Code executed at most once is not hot.  */
   if (count <= MAX (profile_info ? profile_info->runs : 1, 1))
     return false;
@@ -192,9 +196,7 @@ bool
 maybe_hot_bb_p (struct function *fun, const_basic_block bb)
 {
   gcc_checking_assert (fun);
-  if (!maybe_hot_count_p (fun, bb->count))
-    return false;
-  return maybe_hot_frequency_p (fun, bb->frequency);
+  return maybe_hot_count_p (fun, bb->count);
 }
 
 /* Return true in case BB can be CPU intensive and should be optimized
@@ -203,9 +205,7 @@ maybe_hot_bb_p (struct function *fun, const_basic_block bb)
 bool
 maybe_hot_edge_p (edge e)
 {
-  if (!maybe_hot_count_p (cfun, e->count ()))
-    return false;
-  return maybe_hot_frequency_p (cfun, EDGE_FREQUENCY (e));
+  return maybe_hot_count_p (cfun, e->count ());
 }
 
 /* Return true if profile COUNT and FREQUENCY, or function FUN static
@@ -213,7 +213,7 @@ maybe_hot_edge_p (edge e)
    
 static bool
 probably_never_executed (struct function *fun,
-                         profile_count count, int)
+                         profile_count count)
 {
   gcc_checking_assert (fun);
   if (count == profile_count::zero ())
@@ -238,7 +238,7 @@ probably_never_executed (struct function *fun,
 bool
 probably_never_executed_bb_p (struct function *fun, const_basic_block bb)
 {
-  return probably_never_executed (fun, bb->count, bb->frequency);
+  return probably_never_executed (fun, bb->count);
 }
 
 
@@ -259,7 +259,7 @@ probably_never_executed_edge_p (struct function *fun, edge e)
 {
   if (unlikely_executed_edge_p (e))
     return true;
-  return probably_never_executed (fun, e->count (), EDGE_FREQUENCY (e));
+  return probably_never_executed (fun, e->count ());
 }
 
 /* Return true when current function should always be optimized for size.  */
@@ -1289,7 +1289,8 @@ combine_predictions_for_bb (basic_block bb, bool dry_run)
     }
   clear_bb_predictions (bb);
 
-  if (!bb->count.initialized_p () && !dry_run)
+  if ((!bb->count.nonzero_p () || !first->probability.initialized_p ())
+      && !dry_run)
     {
       first->probability
 	 = profile_probability::from_reg_br_prob_base (combined_probability);
@@ -3014,10 +3015,7 @@ propagate_freq (basic_block head, bitmap tovisit)
       BLOCK_INFO (bb)->npredecessors = count;
       /* When function never returns, we will never process exit block.  */
       if (!count && bb == EXIT_BLOCK_PTR_FOR_FN (cfun))
-	{
-	  bb->count = profile_count::zero ();
-	  bb->frequency = 0;
-	}
+	bb->count = profile_count::zero ();
     }
 
   BLOCK_INFO (head)->frequency = 1;
@@ -3050,7 +3048,10 @@ propagate_freq (basic_block head, bitmap tovisit)
 				  * BLOCK_INFO (e->src)->frequency /
 				  REG_BR_PROB_BASE);  */
 
-		sreal tmp = e->probability.to_reg_br_prob_base ();
+		/* FIXME: Graphite is producing edges with no profile. Once
+		   this is fixed, drop this.  */
+		sreal tmp = e->probability.initialized_p () ?
+			    e->probability.to_reg_br_prob_base () : 0;
 		tmp *= BLOCK_INFO (e->src)->frequency;
 		tmp *= real_inv_br_prob_base;
 		frequency += tmp;
@@ -3082,7 +3083,10 @@ propagate_freq (basic_block head, bitmap tovisit)
 	     = ((e->probability * BLOCK_INFO (bb)->frequency)
 	     / REG_BR_PROB_BASE); */
 
-	  sreal tmp = e->probability.to_reg_br_prob_base ();
+	  /* FIXME: Graphite is producing edges with no profile. Once
+	     this is fixed, drop this.  */
+	  sreal tmp = e->probability.initialized_p () ?
+		      e->probability.to_reg_br_prob_base () : 0;
 	  tmp *= BLOCK_INFO (bb)->frequency;
 	  EDGE_INFO (e)->back_edge_prob = tmp * real_inv_br_prob_base;
 	}
@@ -3196,10 +3200,26 @@ drop_profile (struct cgraph_node *node, profile_count call_count)
     }
 
   basic_block bb;
-  FOR_ALL_BB_FN (bb, fn)
+  push_cfun (DECL_STRUCT_FUNCTION (node->decl));
+  if (flag_guess_branch_prob)
     {
-      bb->count = profile_count::uninitialized ();
+      bool clear_zeros
+	 = ENTRY_BLOCK_PTR_FOR_FN
+		 (DECL_STRUCT_FUNCTION (node->decl))->count.nonzero_p ();
+      FOR_ALL_BB_FN (bb, fn)
+	if (clear_zeros || !(bb->count == profile_count::zero ()))
+	  bb->count = bb->count.guessed_local ();
+      DECL_STRUCT_FUNCTION (node->decl)->cfg->count_max =
+        DECL_STRUCT_FUNCTION (node->decl)->cfg->count_max.guessed_local ();
     }
+  else
+    {
+      FOR_ALL_BB_FN (bb, fn)
+	bb->count = profile_count::uninitialized ();
+      DECL_STRUCT_FUNCTION (node->decl)->cfg->count_max
+	 = profile_count::uninitialized ();
+    }
+  pop_cfun ();
 
   struct cgraph_edge *e;
   for (e = node->callees; e; e = e->next_caller)
@@ -3300,33 +3320,16 @@ handle_missing_profiles (void)
 bool
 counts_to_freqs (void)
 {
-  gcov_type count_max;
-  profile_count true_count_max = profile_count::zero ();
+  profile_count true_count_max = profile_count::uninitialized ();
   basic_block bb;
 
-  /* Don't overwrite the estimated frequencies when the profile for
-     the function is missing.  We may drop this function PROFILE_GUESSED
-     later in drop_profile ().  */
-  if (!ENTRY_BLOCK_PTR_FOR_FN (cfun)->count.initialized_p ()
-      || ENTRY_BLOCK_PTR_FOR_FN (cfun)->count == profile_count::zero ())
-    return false;
-
   FOR_BB_BETWEEN (bb, ENTRY_BLOCK_PTR_FOR_FN (cfun), NULL, next_bb)
-    if (bb->count > true_count_max)
-      true_count_max = bb->count;
-
-  /* If we have no counts to base frequencies on, keep those that are
-     already there.  */
-  if (!(true_count_max > 0))
-    return false;
+    if (!(bb->count < true_count_max))
+      true_count_max = true_count_max.max (bb->count);
 
-  count_max = true_count_max.to_gcov_type ();
-
-  FOR_ALL_BB_FN (bb, cfun)
-    if (bb->count.initialized_p ())
-      bb->frequency = RDIV (bb->count.to_gcov_type () * BB_FREQ_MAX, count_max);
+  cfun->cfg->count_max = true_count_max;
 
-  return true;
+  return true_count_max.nonzero_p ();
 }
 
 /* Return true if function is likely to be expensive, so there is no point to
@@ -3348,11 +3351,11 @@ expensive_function_p (int threshold)
   /* Frequencies are out of range.  This either means that function contains
      internal loop executing more than BB_FREQ_MAX times or profile feedback
      is available and function has not been executed at all.  */
-  if (ENTRY_BLOCK_PTR_FOR_FN (cfun)->frequency == 0)
+  if (ENTRY_BLOCK_PTR_FOR_FN (cfun)->count.to_frequency (cfun) == 0)
     return true;
 
   /* Maximally BB_FREQ_MAX^2 so overflow won't happen.  */
-  limit = ENTRY_BLOCK_PTR_FOR_FN (cfun)->frequency * threshold;
+  limit = ENTRY_BLOCK_PTR_FOR_FN (cfun)->count.to_frequency (cfun) * threshold;
   FOR_EACH_BB_FN (bb, cfun)
     {
       rtx_insn *insn;
@@ -3360,7 +3363,7 @@ expensive_function_p (int threshold)
       FOR_BB_INSNS (bb, insn)
 	if (active_insn_p (insn))
 	  {
-	    sum += bb->frequency;
+	    sum += bb->count.to_frequency (cfun);
 	    if (sum > limit)
 	      return true;
 	}
@@ -3409,7 +3412,6 @@ propagate_unlikely_bbs_forward (void)
 		     "Basic block %i is marked unlikely by forward prop\n",
 		     bb->index);
 	  bb->count = profile_count::zero ();
-	  bb->frequency = 0;
 	}
       else
         bb->aux = NULL;
@@ -3440,9 +3442,6 @@ determine_unlikely_bbs ()
 	  bb->count = profile_count::zero ();
 	}
 
-      if (bb->count == profile_count::zero ())
-        bb->frequency = 0;
-
       FOR_EACH_EDGE (e, ei, bb->succs)
 	if (!(e->probability == profile_probability::never ())
 	    && unlikely_executed_edge_p (e))
@@ -3497,7 +3496,6 @@ determine_unlikely_bbs ()
 		 "Basic block %i is marked unlikely by backward prop\n",
 		 bb->index);
       bb->count = profile_count::zero ();
-      bb->frequency = 0;
       FOR_EACH_EDGE (e, ei, bb->preds)
 	if (!(e->probability == profile_probability::never ()))
 	  {
@@ -3554,8 +3552,13 @@ estimate_bb_frequencies (bool force)
 
 	  FOR_EACH_EDGE (e, ei, bb->succs)
 	    {
-	      EDGE_INFO (e)->back_edge_prob
-		 = e->probability.to_reg_br_prob_base ();
+	      /* FIXME: Graphite is producing edges with no profile. Once
+		 this is fixed, drop this.  */
+	      if (e->probability.initialized_p ())
+	        EDGE_INFO (e)->back_edge_prob
+		   = e->probability.to_reg_br_prob_base ();
+	      else
+		EDGE_INFO (e)->back_edge_prob = REG_BR_PROB_BASE / 2;
 	      EDGE_INFO (e)->back_edge_prob *= real_inv_br_prob_base;
 	    }
 	}
@@ -3564,16 +3567,28 @@ estimate_bb_frequencies (bool force)
          to outermost to examine frequencies for back edges.  */
       estimate_loops ();
 
+      bool global0 = ENTRY_BLOCK_PTR_FOR_FN (cfun)->count.initialized_p ()
+		     && ENTRY_BLOCK_PTR_FOR_FN (cfun)->count.ipa_p ();
+
       freq_max = 0;
       FOR_EACH_BB_FN (bb, cfun)
 	if (freq_max < BLOCK_INFO (bb)->frequency)
 	  freq_max = BLOCK_INFO (bb)->frequency;
 
       freq_max = real_bb_freq_max / freq_max;
+      cfun->cfg->count_max = profile_count::uninitialized ();
       FOR_BB_BETWEEN (bb, ENTRY_BLOCK_PTR_FOR_FN (cfun), NULL, next_bb)
 	{
 	  sreal tmp = BLOCK_INFO (bb)->frequency * freq_max + real_one_half;
-	  bb->frequency = tmp.to_int ();
+	  profile_count count = profile_count::from_gcov_type (tmp.to_int ());	
+
+	  /* If we have profile feedback in which this function was never
+	     executed, then preserve this info.  */
+	  if (global0)
+	    bb->count = count.global0 ();
+	  else if (!(bb->count == profile_count::zero ()))
+	    bb->count = count.guessed_local ();
+          cfun->cfg->count_max = cfun->cfg->count_max.max (bb->count);
 	}
 
       free_aux_for_blocks ();
@@ -3598,7 +3613,8 @@ compute_function_frequency (void)
   if (profile_status_for_fn (cfun) != PROFILE_READ)
     {
       int flags = flags_from_decl_or_type (current_function_decl);
-      if (ENTRY_BLOCK_PTR_FOR_FN (cfun)->count == profile_count::zero ()
+      if ((ENTRY_BLOCK_PTR_FOR_FN (cfun)->count.ipa_p ()
+	   && ENTRY_BLOCK_PTR_FOR_FN (cfun)->count.ipa() == profile_count::zero ())
 	  || lookup_attribute ("cold", DECL_ATTRIBUTES (current_function_decl))
 	     != NULL)
 	{
@@ -3717,7 +3733,7 @@ pass_profile::execute (function *fun)
    {
      struct loop *loop;
      FOR_EACH_LOOP (loop, LI_FROM_INNERMOST)
-       if (loop->header->frequency)
+       if (loop->header->count.initialized_p ())
          fprintf (dump_file, "Loop got predicted %d to iterate %i times.\n",
        	   loop->num,
        	   (int)expected_loop_iterations_unbounded (loop));
@@ -3843,15 +3859,12 @@ rebuild_frequencies (void)
      which may also lead to frequencies incorrectly reduced to 0. There
      is less precision in the probabilities, so we only do this for small
      max counts.  */
-  profile_count count_max = profile_count::zero ();
+  cfun->cfg->count_max = profile_count::uninitialized ();
   basic_block bb;
   FOR_BB_BETWEEN (bb, ENTRY_BLOCK_PTR_FOR_FN (cfun), NULL, next_bb)
-    if (bb->count > count_max)
-      count_max = bb->count;
+    cfun->cfg->count_max = cfun->cfg->count_max.max (bb->count);
 
-  if (profile_status_for_fn (cfun) == PROFILE_GUESSED
-      || (!flag_auto_profile && profile_status_for_fn (cfun) == PROFILE_READ
-	  && count_max < REG_BR_PROB_BASE / 10))
+  if (profile_status_for_fn (cfun) == PROFILE_GUESSED)
     {
       loop_optimizer_init (0);
       add_noreturn_fake_exit_edges ();
@@ -4017,17 +4030,19 @@ force_edge_cold (edge e, bool impossible)
 	 after loop transforms.  */
       if (!(prob_sum > profile_probability::never ())
 	  && count_sum == profile_count::zero ()
-	  && single_pred_p (e->src) && e->src->frequency > (impossible ? 0 : 1))
+	  && single_pred_p (e->src) && e->src->count.to_frequency (cfun)
+	     > (impossible ? 0 : 1))
 	{
-	  int old_frequency = e->src->frequency;
+	  int old_frequency = e->src->count.to_frequency (cfun);
 	  if (dump_file && (dump_flags & TDF_DETAILS))
 	    fprintf (dump_file, "Making bb %i %s.\n", e->src->index,
 		     impossible ? "impossible" : "cold");
-	  e->src->frequency = MIN (e->src->frequency, impossible ? 0 : 1);
+	  int new_frequency = MIN (e->src->count.to_frequency (cfun),
+				   impossible ? 0 : 1);
 	  if (impossible)
 	    e->src->count = profile_count::zero ();
 	  else
-	    e->src->count = e->count ().apply_scale (e->src->frequency,
+	    e->src->count = e->count ().apply_scale (new_frequency,
 						     old_frequency);
 	  force_edge_cold (single_pred_edge (e->src), impossible);
 	}