asan.c (asan_emit_stack_protection): Update.

	* asan.c (asan_emit_stack_protection): Update.
	(create_cond_insert_point): Update.
	* auto-profile.c (afdo_propagate_circuit): Update.
	* basic-block.h (struct edge_def): Turn probability to
	profile_probability.
	(EDGE_FREQUENCY): Update.
	* bb-reorder.c (find_traces_1_round): Update.
	(better_edge_p): Update.
	(sanitize_hot_paths): Update.
	* cfg.c (unchecked_make_edge): Initialize probability to uninitialized.
	(make_single_succ_edge): Update.
	(check_bb_profile): Update.
	(dump_edge_info): Update.
	(update_bb_profile_for_threading): Update.
	* cfganal.c (connect_infinite_loops_to_exit): Initialize new edge
	probabilitycount to 0.
	* cfgbuild.c (compute_outgoing_frequencies): Update.
	* cfgcleanup.c (try_forward_edges): Update.
	(outgoing_edges_match): Update.
	(try_crossjump_to_edge): Update.
	* cfgexpand.c (expand_gimple_cond): Update make_single_succ_edge.
	(expand_gimple_tailcall): Update.
	(construct_init_block): Use make_single_succ_edge.
	(construct_exit_block): Use make_single_succ_edge.
	* cfghooks.c (verify_flow_info): Update.
	(redirect_edge_succ_nodup): Update.
	(split_edge): Update.
	(account_profile_record): Update.
	* cfgloopanal.c (single_likely_exit): Update.
	* cfgloopmanip.c (scale_loop_profile): Update.
	(set_zero_probability): Remove.
	(duplicate_loop_to_header_edge): Update.
	* cfgloopmanip.h (loop_version): Update prototype.
	* cfgrtl.c (try_redirect_by_replacing_jump): Update.
	(force_nonfallthru_and_redirect): Update.
	(update_br_prob_note): Update.
	(rtl_verify_edges): Update.
	(purge_dead_edges): Update.
	(rtl_lv_add_condition_to_bb): Update.
	* cgraph.c: (cgraph_edge::redirect_call_stmt_to_calle): Update.
	* cgraphunit.c (init_lowered_empty_function): Update.
	(cgraph_node::expand_thunk): Update.
	* cilk-common.c: Include profile-count.h
	* dojump.c (inv): Remove.
	(jumpifnot): Update.
	(jumpifnot_1): Update.
	(do_jump_1): Update.
	(do_jump): Update.
	(do_jump_by_parts_greater_rtx): Update.
	(do_compare_rtx_and_jump): Update.
	* dojump.h (jumpifnot, jumpifnot_1, jumpif_1, jumpif, do_jump,
	do_jump_1. do_compare_rtx_and_jump): Update prototype.
	* dwarf2cfi.c: Include profile-count.h
	* except.c (dw2_build_landing_pads): Use make_single_succ_edge.
	(sjlj_emit_dispatch_table): Likewise.
	* explow.c: Include profile-count.h
	* expmed.c (emit_store_flag_force): Update.
	(do_cmp_and_jump): Update.
	* expr.c (compare_by_pieces_d::generate): Update.
	(compare_by_pieces_d::finish_mode): Update.
	(emit_block_move_via_loop): Update.
	(store_expr_with_bounds): Update.
	(store_constructor): Update.
	(expand_expr_real_2): Update.
	(expand_expr_real_1): Update.
	* expr.h (try_casesi, try_tablejump): Update prototypes.
	* gimple-pretty-print.c (dump_probability): Update.
	(dump_profile): New.
	(dump_gimple_label): Update.
	(dump_gimple_bb_header): Update.
	* graph.c (draw_cfg_node_succ_edges): Update.
	* hsa-gen.c (convert_switch_statements): Update.
	* ifcvt.c (cheap_bb_rtx_cost_p): Update.
	(find_if_case_1): Update.
	(find_if_case_2): Update.
	* internal-fn.c (expand_arith_overflow_result_store): Update.
	(expand_addsub_overflow): Update.
	(expand_neg_overflow): Update.
	(expand_mul_overflow): Update.
	(expand_vector_ubsan_overflow): Update.
	* ipa-cp.c (good_cloning_opportunity_p): Update.
	* ipa-split.c (split_function): Use make_single_succ_edge.
	* ipa-utils.c (ipa_merge_profiles): Update.
	* loop-doloop.c (add_test): Update.
	(doloop_modify): Update.
	* loop-unroll.c (compare_and_jump_seq): Update.
	(unroll_loop_runtime_iterations): Update.
	* lra-constraints.c (lra_inheritance): Update.
	* lto-streamer-in.c (input_cfg): Update.
	* lto-streamer-out.c (output_cfg): Update.
	* mcf.c (adjust_cfg_counts): Update.
	* modulo-sched.c (sms_schedule): Update.
	* omp-expand.c (expand_omp_for_init_counts): Update.
	(extract_omp_for_update_vars): Update.
	(expand_omp_ordered_sink): Update.
	(expand_omp_for_ordered_loops): Update.
	(expand_omp_for_generic): Update.
	(expand_omp_for_static_nochunk): Update.
	(expand_omp_for_static_chunk): Update.
	(expand_cilk_for): Update.
	(expand_omp_simd): Update.
	(expand_omp_taskloop_for_outer): Update.
	(expand_omp_taskloop_for_inner): Update.
	* omp-simd-clone.c (simd_clone_adjust): Update.
	* optabs.c (expand_doubleword_shift): Update.
	(expand_abs): Update.
	(emit_cmp_and_jump_insn_1): Update.
	(expand_compare_and_swap_loop): Update.
	* optabs.h (emit_cmp_and_jump_insns): Update prototype.
	* predict.c (predictable_edge_p): Update.
	(edge_probability_reliable_p): Update.
	(set_even_probabilities): Update.
	(combine_predictions_for_insn): Update.
	(combine_predictions_for_bb): Update.
	(propagate_freq): Update.
	(estimate_bb_frequencies): Update.
	(force_edge_cold): Update.
	* profile-count.c (profile_count::dump): Add missing space into dump.
	(profile_count::debug): Add newline.
	(profile_count::differs_from_p): Explicitly convert to unsigned.
	(profile_count::stream_in): Update.
	(profile_probability::dump): New member function.
	(profile_probability::debug): New member function.
	(profile_probability::differs_from_p): New member function.
	(profile_probability::differs_lot_from_p): New member function.
	(profile_probability::stream_in): New member function.
	(profile_probability::stream_out): New member function.
	* profile-count.h (profile_count_quality): Rename to ...
	(profile_quality): ... this one.
	(profile_probability): New.
	(profile_count): Update.
	* profile.c (compute_branch_probabilities): Update.
	* recog.c (peep2_attempt): Update.
	* sched-ebb.c (schedule_ebbs): Update.
	* sched-rgn.c (find_single_block_region): Update.
	(compute_dom_prob_ps): Update.
	(schedule_region): Update.
	* sel-sched-ir.c (compute_succs_info): Update.
	* stmt.c (struct case_node): Update.
	(do_jump_if_equal): Update.
	(get_outgoing_edge_probs): Update.
	(conditional_probability): Update.
	(emit_case_dispatch_table): Update.
	(expand_case): Update.
	(expand_sjlj_dispatch_table): Update.
	(emit_case_nodes): Update.
	* targhooks.c: Update.
	* tracer.c (better_p): Update.
	(find_best_successor): Update.
	* trans-mem.c (expand_transaction): Update.
	* tree-call-cdce.c: Update.
	* tree-cfg.c (gimple_split_edge): Upate.
	(move_sese_region_to_fn): Upate.
	* tree-cfgcleanup.c (cleanup_control_expr_graph): Upate.
	* tree-eh.c (lower_resx): Upate.
	(cleanup_empty_eh_move_lp): Upate.
	* tree-if-conv.c (version_loop_for_if_conversion): Update.
	* tree-inline.c (copy_edges_for_bb): Update.
	(copy_cfg_body): Update.
	* tree-parloops.c (gen_parallel_loop): Update.
	* tree-profile.c (gimple_gen_ic_func_profiler): Update.
	(gimple_gen_time_profiler): Update.
	* tree-ssa-dce.c (remove_dead_stmt): Update.
	* tree-ssa-ifcombine.c (update_profile_after_ifcombine): Update.
	* tree-ssa-loop-im.c (execute_sm_if_changed): Update.
	* tree-ssa-loop-ivcanon.c (remove_exits_and_undefined_stmts): Update.
	(unloop_loops): Update.
	(try_peel_loop): Update.
	* tree-ssa-loop-manip.c (tree_transform_and_unroll_loop): Update.
	* tree-ssa-loop-split.c (connect_loops): Update.
	(split_loop): Update.
	* tree-ssa-loop-unswitch.c (tree_unswitch_loop): Update.
	(hoist_guard): Update.
	* tree-ssa-phionlycprop.c (propagate_rhs_into_lhs): Update.
	* tree-ssa-phiopt.c (replace_phi_edge_with_variable): Update.
	(value_replacement): Update.
	* tree-ssa-reassoc.c (branch_fixup): Update.
	* tree-ssa-tail-merge.c (replace_block_by): Update.
	* tree-ssa-threadupdate.c (remove_ctrl_stmt_and_useless_edges): Update.
	(create_edge_and_update_destination_phis): Update.
	(compute_path_counts): Update.
	(recompute_probabilities): Update.
	(update_joiner_offpath_counts): Update.
	(freqs_to_counts_path): Update.
	(duplicate_thread_path): Update.
	* tree-switch-conversion.c (hoist_edge_and_branch_if_true): Update.
	(struct switch_conv_info): Update.
	(gen_inbound_check): Update.
	* tree-vect-loop-manip.c (slpeel_add_loop_guard): Update.
	(vect_do_peeling): Update.
	(vect_loop_versioning): Update.
	* tree-vect-loop.c (scale_profile_for_vect_loop): Update.
	(optimize_mask_stores): Update.
	* ubsan.c (ubsan_expand_null_ifn): Update.
	* value-prof.c (gimple_divmod_fixed_value): Update.
	(gimple_divmod_fixed_value_transform): Update.
	(gimple_mod_pow2): Update.
	(gimple_mod_pow2_value_transform): Update.
	(gimple_mod_subtract): Update.
	(gimple_mod_subtract_transform): Update.
	(gimple_ic): Update.
	(gimple_stringop_fixed_value): Update.
	(gimple_stringops_transform): Update.
	* value-prof.h: Update.

From-SVN: r249800

1 files changed, 458 insertions, 22 deletions

diff --git a/gcc/profile-count.h b/gcc/profile-count.h
index 0f77e4e..461dac6 100644
--- a/gcc/profile-count.h
+++ b/gcc/profile-count.h
@@ -23,19 +23,19 @@ along with GCC; see the file COPYING3.  If not see
 
 /* Quality of the proflie count.  Because gengtype does not support enums
    inside of clases, this is in global namespace.  */
-enum profile_count_quality {
+enum profile_quality {
   /* Profile is based on static branch prediction heuristics.  It may or may
      not reflect the reality.  */
-  count_guessed = 0,
+  profile_guessed = 0,
   /* Profile was determined by autofdo.  */
-  count_afdo = 1,
+  profile_afdo = 1,
   /* Profile was originally based on feedback but it was adjusted 
      by code duplicating optimization.  It may not precisely reflect the
      particular code path.  */
-  count_adjusted = 2,
+  profile_adjusted = 2,
   /* Profile was read from profile feedback or determined by accurate static
      method.  */
-  count_read = 3
+  profile_precise = 3
 };
 
 /* The base value for branch probability notes and edge probabilities.  */
@@ -43,6 +43,422 @@ enum profile_count_quality {
 
 #define RDIV(X,Y) (((X) + (Y) / 2) / (Y))
 
+/* Data type to hold probabilities.  It implement fixed point arithmetics
+   with capping so probability is always in range [0,1] and scaling requiring
+   values greater than 1 needs to be represented otherwise.
+
+   In addition to actual value the quality of profile is tracked and propagated
+   through all operations.  Special value UNINITIALIZED is used for probabilities
+   that has not been detemrined yet (for example bacause of
+   -fno-guess-branch-probability)
+
+   Typically probabilities are derived from profile feedback (via
+   probability_in_gcov_type), autoFDO or guessed statically and then propagated
+   thorough the compilation.
+
+   Named probabilities are available:
+     - never           (0 probability)
+     - guessed_never
+     - very_unlikely   (1/2000 probability)
+     - unlikely        (1/5 probablity)
+     - even            (1/2 probability)
+     - likely          (4/5 probability)
+     - very_likely     (1999/2000 probability)
+     - guessed_always
+     - always
+
+   Named probabilities except for never/always are assumed to be statically
+   guessed and thus not necessarily acurate.  The difference between never
+   and guessedn never is that the first one should be used only in case that
+   well behaving program will very likely not execute the "never" path.
+   For example if the path is going to abort () call or it exception handling.
+
+   Alawyas and guessted_always probabilities are symmetric.
+
+   For legacy code we support conversion to/from REG_BR_PROB_BASE based fixpoint
+   integer arithmetics. Once the code is converted to branch probabiitlies,
+   these conversions will probably go away because they are lossy.
+*/
+
+class GTY((user)) profile_probability
+{
+  /* For now use values in range 0...REG_BR_PROB_BASE.  Later we can use full
+     precision of 30 bits available.  */
+
+  static const int n_bits = 30;
+  static const uint32_t max_probability = REG_BR_PROB_BASE;
+  static const uint32_t uninitialized_probability = ((uint32_t) 1 << n_bits) - 1;
+
+  uint32_t m_val : 30;
+  enum profile_quality m_quality : 2;
+
+  friend class profile_count;
+public:
+
+  /* Named probabilities.  */
+  static profile_probability never ()
+    {
+      profile_probability ret;
+      ret.m_val = 0;
+      ret.m_quality = profile_precise;
+      return ret;
+    }
+  static profile_probability guessed_never ()
+    {
+      profile_probability ret;
+      ret.m_val = 0;
+      ret.m_quality = profile_guessed;
+      return ret;
+    }
+  static profile_probability very_unlikely ()
+    {
+      /* Be consistent with PROB_VERY_UNLIKELY in predict.h.  */
+      profile_probability r
+	 = profile_probability::always ().apply_scale (1, 2000);
+      r.m_val--;
+      return r;
+    }
+  static profile_probability unlikely ()
+    {
+      /* Be consistent with PROB_VERY_LIKELY in predict.h.  */
+      profile_probability r
+	 = profile_probability::always ().apply_scale (1, 5);
+      r.m_val--;
+      return r;
+    }
+  static profile_probability even ()
+    {
+      return profile_probability::always ().apply_scale (1, 2);
+    }
+  static profile_probability very_likely ()
+    {
+      return profile_probability::always () - very_unlikely ();
+    }
+  static profile_probability likely ()
+    {
+      return profile_probability::always () - unlikely ();
+    }
+  static profile_probability guessed_always ()
+    {
+      profile_probability ret;
+      ret.m_val = max_probability;
+      ret.m_quality = profile_guessed;
+      return ret;
+    }
+  static profile_probability always ()
+    {
+      profile_probability ret;
+      ret.m_val = max_probability;
+      ret.m_quality = profile_precise;
+      return ret;
+    }
+  /* Probabilities which has not been initialized. Either because
+     initialization did not happen yet or because profile is unknown.  */
+  static profile_probability uninitialized ()
+    {
+      profile_probability c;
+      c.m_val = uninitialized_probability;
+      c.m_quality = profile_guessed;
+      return c;
+    }
+
+
+  /* Return true if value has been initialized.  */
+  bool initialized_p () const
+    {
+      return m_val != uninitialized_probability;
+    }
+  /* Return true if value can be trusted.  */
+  bool reliable_p () const
+    {
+      return initialized_p ();
+    }
+
+  /* Conversion from and to REG_BR_PROB_BASE integer fixpoint arithmetics.
+     this is mostly to support legacy code and hsould go away.  */
+  static profile_probability from_reg_br_prob_base (int v)
+    {
+      profile_probability ret;
+      gcc_checking_assert (v >= 0 && v <= REG_BR_PROB_BASE);
+      ret.m_val = RDIV (v * max_probability, REG_BR_PROB_BASE);
+      ret.m_quality = profile_guessed;
+      return ret;
+    }
+  int to_reg_br_prob_base () const
+    {
+      gcc_checking_assert (initialized_p ());
+      return RDIV (m_val * REG_BR_PROB_BASE, max_probability);
+    }
+
+  /* Return VAL1/VAL2.  */
+  static profile_probability probability_in_gcov_type
+				 (gcov_type val1, gcov_type val2)
+    {
+      profile_probability ret;
+      gcc_checking_assert (val1 >= 0 && val2 > 0);
+      if (val1 > val2)
+	ret.m_val = max_probability;
+      else
+        ret.m_val = RDIV (val1 * max_probability, val2);
+      ret.m_quality = profile_precise;
+      return ret;
+    }
+
+  /* Basic operations.  */
+  bool operator== (const profile_probability &other) const
+    {
+      return m_val == other.m_val && m_quality == other.m_quality;
+    }
+  profile_probability operator+ (const profile_probability &other) const
+    {
+      if (other == profile_probability::never ())
+	return *this;
+      if (*this == profile_probability::never ())
+	return other;
+      if (!initialized_p () || !other.initialized_p ())
+	return profile_probability::uninitialized ();
+
+      profile_probability ret;
+      ret.m_val = MIN ((uint32_t)(m_val + other.m_val), max_probability);
+      ret.m_quality = MIN (m_quality, other.m_quality);
+      return ret;
+    }
+  profile_probability &operator+= (const profile_probability &other)
+    {
+      if (other == profile_probability::never ())
+	return *this;
+      if (*this == profile_probability::never ())
+	{
+	  *this = other;
+	  return *this;
+	}
+      if (!initialized_p () || !other.initialized_p ())
+	return *this = profile_probability::uninitialized ();
+      else
+	{
+	  m_val = MIN ((uint32_t)(m_val + other.m_val), max_probability);
+          m_quality = MIN (m_quality, other.m_quality);
+	}
+      return *this;
+    }
+  profile_probability operator- (const profile_probability &other) const
+    {
+      if (*this == profile_probability::never ()
+	  || other == profile_probability::never ())
+	return *this;
+      if (!initialized_p () || !other.initialized_p ())
+	return profile_probability::uninitialized ();
+      profile_probability ret;
+      ret.m_val = m_val >= other.m_val ? m_val - other.m_val : 0;
+      ret.m_quality = MIN (m_quality, other.m_quality);
+      return ret;
+    }
+  profile_probability &operator-= (const profile_probability &other)
+    {
+      if (*this == profile_probability::never ()
+	  || other == profile_probability::never ())
+	return *this;
+      if (!initialized_p () || !other.initialized_p ())
+	return *this = profile_probability::uninitialized ();
+      else
+	{
+	  m_val = m_val >= other.m_val ? m_val - other.m_val : 0;
+          m_quality = MIN (m_quality, other.m_quality);
+	}
+      return *this;
+    }
+  profile_probability operator* (const profile_probability &other) const
+    {
+      if (*this == profile_probability::never ()
+	  || other == profile_probability::never ())
+	return profile_probability::never ();
+      if (!initialized_p () || !other.initialized_p ())
+	return profile_probability::uninitialized ();
+      profile_probability ret;
+      ret.m_val = RDIV ((uint64_t)m_val * other.m_val, max_probability);
+      ret.m_quality = MIN (m_quality, other.m_quality);
+      return ret;
+    }
+  profile_probability &operator*= (const profile_probability &other)
+    {
+      if (*this == profile_probability::never ()
+	  || other == profile_probability::never ())
+	return *this = profile_probability::never ();
+      if (!initialized_p () || !other.initialized_p ())
+	return *this = profile_probability::uninitialized ();
+      else
+	{
+          m_val = RDIV ((uint64_t)m_val * other.m_val, max_probability);
+          m_quality = MIN (m_quality, other.m_quality);
+	}
+      return *this;
+    }
+  profile_probability operator/ (const profile_probability &other) const
+    {
+      if (*this == profile_probability::never ())
+	return profile_probability::never ();
+      if (!initialized_p () || !other.initialized_p ())
+	return profile_probability::uninitialized ();
+      profile_probability ret;
+      if (m_val >= other.m_val)
+	ret.m_val = max_probability;
+      else if (!m_val)
+	ret.m_val = 0;
+      else
+	{
+	  gcc_checking_assert (other.m_val);
+          ret.m_val = MIN (RDIV ((uint64_t)m_val * max_probability,
+				 other.m_val),
+			   max_probability);
+	}
+      ret.m_quality = MIN (m_quality, other.m_quality);
+      return ret;
+    }
+  profile_probability &operator/= (const profile_probability &other)
+    {
+      if (*this == profile_probability::never ())
+	return *this = profile_probability::never ();
+      if (!initialized_p () || !other.initialized_p ())
+	return *this = profile_probability::uninitialized ();
+      else
+	{
+	  if (m_val > other.m_val)
+	    m_val = max_probability;
+	  else if (!m_val)
+	    ;
+	  else
+	    {
+	      gcc_checking_assert (other.m_val);
+              m_val = MIN (RDIV ((uint64_t)m_val * max_probability,
+				 other.m_val),
+			   max_probability);
+	    }
+          m_quality = MIN (m_quality, other.m_quality);
+	}
+      return *this;
+    }
+
+  gcov_type apply (gcov_type val) const
+    {
+      if (*this == profile_probability::uninitialized ())
+	return val / 2;
+      return RDIV (val * m_val, max_probability);
+    }
+
+  /* Return 1-*THIS.  */
+  profile_probability invert () const
+    {
+      return profile_probability::always() - *this;
+    }
+
+  profile_probability combine_with_freq (int freq1, profile_probability other,
+					 int freq2) const
+    {
+      profile_probability ret;
+
+      if (*this == profile_probability::uninitialized ()
+	  || other == profile_probability::uninitialized ())
+	return profile_probability::uninitialized ();
+
+      gcc_checking_assert (freq1 >= 0 && freq2 >= 0);
+      if (!freq1 && !freq2)
+	{
+	  ret.m_val = (m_val + other.m_val) / 2;
+	}
+      else
+	ret.m_val = RDIV (m_val * (uint64_t) freq1
+			  + other.m_val * (uint64_t) freq2, freq1 + freq2);
+      ret.m_quality = MIN (m_quality, other.m_quality);
+      return ret;
+    }
+
+  /* Return *THIS * NUM / DEN.  */
+  profile_probability apply_scale (int64_t num, int64_t den) const
+    {
+      if (*this == profile_probability::never ())
+	return *this;
+      if (!initialized_p ())
+	return profile_probability::uninitialized ();
+      profile_probability ret;
+      ret.m_val = MIN (RDIV (m_val * num, den),
+		       max_probability);
+      ret.m_quality = MIN (m_quality, profile_adjusted);
+      return ret;
+    }
+
+  /* Return true when the probability of edge is reliable.
+
+     The profile guessing code is good at predicting branch outcome (ie.
+     taken/not taken), that is predicted right slightly over 75% of time.
+     It is however notoriously poor on predicting the probability itself.
+     In general the profile appear a lot flatter (with probabilities closer
+     to 50%) than the reality so it is bad idea to use it to drive optimization
+     such as those disabling dynamic branch prediction for well predictable
+     branches.
+
+     There are two exceptions - edges leading to noreturn edges and edges
+     predicted by number of iterations heuristics are predicted well.  This macro
+     should be able to distinguish those, but at the moment it simply check for
+     noreturn heuristic that is only one giving probability over 99% or bellow
+     1%.  In future we might want to propagate reliability information across the
+     CFG if we find this information useful on multiple places.   */
+
+  bool probably_reliable_p () const
+    {
+      if (m_quality >= profile_adjusted)
+	return true;
+      if (!initialized_p ())
+	return false;
+      return m_val < max_probability / 100
+	     || m_val > max_probability - max_probability / 100;
+    }
+
+  /* Return false if profile_probability is bogus.  */
+  bool verify () const
+    {
+      if (m_val == uninitialized_probability)
+	return m_quality == profile_guessed;
+      else
+	return m_val <= REG_BR_PROB_BASE;
+    }
+
+  /* Comparsions are three-state and conservative.  False is returned if
+     the inequality can not be decided.  */
+  bool operator< (const profile_probability &other) const
+    {
+      return initialized_p () && other.initialized_p () && m_val < other.m_val;
+    }
+  bool operator> (const profile_probability &other) const
+    {
+      return initialized_p () && other.initialized_p () && m_val > other.m_val;
+    }
+
+  bool operator<= (const profile_probability &other) const
+    {
+      return initialized_p () && other.initialized_p () && m_val <= other.m_val;
+    }
+  bool operator>= (const profile_probability &other) const
+    {
+      return initialized_p () && other.initialized_p () && m_val >= other.m_val;
+    }
+
+  /* Output THIS to F.  */
+  void dump (FILE *f) const;
+
+  /* Print THIS to stderr.  */
+  void debug () const;
+
+  /* Return true if THIS is known to differ significantly from OTHER.  */
+  bool differs_from_p (profile_probability other) const;
+  /* Return if difference is greater than 50%.  */
+  bool differs_lot_from_p (profile_probability other) const;
+
+  /* LTO streaming support.  */
+  static profile_probability stream_in (struct lto_input_block *);
+  void stream_out (struct output_block *);
+  void stream_out (struct lto_output_stream *);
+};
+
 /* Main data type to hold profile counters in GCC.  In most cases profile
    counts originate from profile feedback. They are 64bit integers
    representing number of executions during the train run.
@@ -85,7 +501,7 @@ class GTY(()) profile_count
   static const uint64_t uninitialized_count = ((uint64_t) 1 << n_bits) - 1;
 
   uint64_t m_val : n_bits;
-  enum profile_count_quality m_quality : 2;
+  enum profile_quality m_quality : 2;
 
   /* Assume numbers smaller than this to multiply.  This is set to make
      testsuite pass, in future we may implement precise multiplication in higer
@@ -108,7 +524,7 @@ public:
     {
       profile_count c;
       c.m_val = uninitialized_count;
-      c.m_quality = count_guessed;
+      c.m_quality = profile_guessed;
       return c;
     }
 
@@ -120,7 +536,7 @@ public:
       profile_count ret;
       gcc_checking_assert (v >= 0 && (uint64_t) v <= max_count);
       ret.m_val = v;
-      ret.m_quality = count_read;
+      ret.m_quality = profile_precise;
       return ret;
     }
 
@@ -207,7 +623,7 @@ public:
   /* Return false if profile_count is bogus.  */
   bool verify () const
     {
-      return m_val != uninitialized_count || m_quality == count_guessed;
+      return m_val != uninitialized_count || m_quality == profile_guessed;
     }
 
   /* Comparsions are three-state and conservative.  False is returned if
@@ -237,7 +653,7 @@ public:
     }
   bool operator>= (const profile_count &other) const
     {
-      return initialized_p () && m_val >= other.m_val;
+      return initialized_p () && other.initialized_p () && m_val >= other.m_val;
     }
   bool operator<= (const gcov_type other) const
     {
@@ -261,7 +677,23 @@ public:
 	return profile_count::uninitialized ();
       profile_count ret;
       ret.m_val = RDIV (m_val * prob, REG_BR_PROB_BASE);
-      ret.m_quality = MIN (m_quality, count_adjusted);
+      ret.m_quality = MIN (m_quality, profile_adjusted);
+      return ret;
+    }
+
+  /* Scale counter according to PROB.  */
+  profile_count apply_probability (profile_probability prob) const
+    {
+      if (*this == profile_count::zero ())
+	return *this;
+      if (prob == profile_probability::never ())
+	return profile_count::zero ();
+      if (!initialized_p ())
+	return profile_count::uninitialized ();
+      profile_count ret;
+      ret.m_val = RDIV (m_val * prob.m_val,
+			profile_probability::max_probability);
+      ret.m_quality = MIN (m_quality, prob.m_quality);
       return ret;
     }
   /* Return *THIS * NUM / DEN.  */
@@ -277,7 +709,7 @@ public:
       gcc_checking_assert ((num <= REG_BR_PROB_BASE
 			    || den <= REG_BR_PROB_BASE) || 1);
       ret.m_val = RDIV (m_val * num, den);
-      ret.m_quality = MIN (m_quality, count_adjusted);
+      ret.m_quality = MIN (m_quality, profile_adjusted);
       return ret;
     }
   profile_count apply_scale (profile_count num, profile_count den) const
@@ -299,23 +731,27 @@ public:
       else
         ret.m_val = RDIV (m_val * RDIV (num.m_val * max_safe_multiplier,
 					den.m_val), max_safe_multiplier);
-      ret.m_quality = MIN (m_quality, count_adjusted);
+      ret.m_quality = MIN (m_quality, profile_adjusted);
       return ret;
     }
 
   /* Return probability of event with counter THIS within event with counter
      OVERALL.  */
-  int probability_in (profile_count overall)
+  profile_probability probability_in (const profile_count overall) const
     {
       if (!m_val)
-	return 0;
-      if (!initialized_p () || !overall.initialized_p ())
-	return REG_BR_PROB_BASE / 2;
-      if (overall < *this)
-	return REG_BR_PROB_BASE;
-      if (!overall.m_val)
-	return REG_BR_PROB_BASE / 2;
-      return RDIV (m_val * REG_BR_PROB_BASE, overall.m_val);
+	return profile_probability::never ();
+      if (!initialized_p () || !overall.initialized_p ()
+	  || !overall.m_val)
+	return profile_probability::uninitialized ();
+      profile_probability ret;
+      if (overall < m_val)
+	ret.m_val = profile_probability::max_probability;
+      else
+        ret.m_val = RDIV (m_val * profile_probability::max_probability,
+			  overall.m_val);
+      ret.m_quality = MIN (m_quality, overall.m_quality);
+      return ret;
     }
 
   /* Output THIS to F.  */