From f1ba88b1b20cb579b3b7ce6ce65470205742be7e Mon Sep 17 00:00:00 2001 From: Xiong Hu Luo Date: Wed, 24 Apr 2019 00:10:44 -0500 Subject: Missed function specialization + partial devirtualization v8: 1. Rebase to master with Martin's static function (r280043) comments merge. Boostrap/testsuite/SPEC2017 tested pass on Power8-LE. 2. TODO: 2.1. C++ devirt for multiple speculative call targets. 2.2. ipa-icf ipa_merge_profiles refine with COMDAT inline testcase. This patch aims to fix PR69678 caused by PGO indirect call profiling performance issues. The bug that profiling data is never working was fixed by Martin's pull back of topN patches, performance got GEOMEAN ~1% improvement(+24% for 511.povray_r specifically). Still, currently the default profile only generates SINGLE indirect target that called more than 75%. This patch leverages MULTIPLE indirect targets use in LTO-WPA and LTO-LTRANS stage, as a result, function specialization, profiling, partial devirtualization, inlining and cloning could be done successfully based on it. Performance can get improved from 0.70 sec to 0.38 sec on simple tests. Details are: 1. PGO with topn is enabled by default now, but only one indirect target edge will be generated in ipa-profile pass, so add variables to enable multiple speculative edges through passes, speculative_id will record the direct edge index bind to the indirect edge, indirect_call_targets length records how many direct edges owned by the indirect edge, postpone gimple_ic to ipa-profile like default as inline pass will decide whether it is benefit to transform indirect call. 2. Use speculative_id to track and search the reference node matched with the direct edge's callee for multiple targets. Actually, it is the caller's responsibility to handle the direct edges mapped to same indirect edge. speculative_call_info will return one of the direct edge specified, this will leverage current IPA edge process framework mostly. 3. Enable LTO WPA/LTRANS stage multiple indirect call targets analysis for profile full support in ipa passes and cgraph_edge functions. speculative_id can be set by make_speculative id when multiple targets are binded to one indirect edge, and cloned if new edge is cloned. speculative_id is streamed out and stream int by lto like lto_stmt_uid. 4. Create and duplicate all speculative direct edge's call summary in ipa-fnsummary.c with auto_vec. 5. Add 1 in module testcase and 2 cross module testcases. 6. Bootstrap and regression test passed on Power8-LE. No function and performance regression for SPEC2017. gcc/ChangeLog 2020-01-14 Xiong Hu Luo PR ipa/69678 * cgraph.c (symbol_table::create_edge): Init speculative_id and target_prob. (cgraph_edge::make_speculative): Add param for setting speculative_id and target_prob. (cgraph_edge::speculative_call_info): Update comments and find reference by speculative_id for multiple indirect targets. (cgraph_edge::resolve_speculation): Decrease the speculations for indirect edge, drop it's speculative if not direct target left. Update comments. (cgraph_edge::redirect_call_stmt_to_callee): Likewise. (cgraph_node::dump): Print num_speculative_call_targets. (cgraph_node::verify_node): Don't report error if speculative edge not include statement. (cgraph_edge::num_speculative_call_targets_p): New function. * cgraph.h (int common_target_id): Remove. (int common_target_probability): Remove. (num_speculative_call_targets): New variable. (make_speculative): Add param for setting speculative_id. (cgraph_edge::num_speculative_call_targets_p): New declare. (target_prob): New variable. (speculative_id): New variable. * ipa-fnsummary.c (analyze_function_body): Create and duplicate call summaries for multiple speculative call targets. * cgraphclones.c (cgraph_node::create_clone): Clone speculative_id. * ipa-profile.c (struct speculative_call_target): New struct. (class speculative_call_summary): New class. (class speculative_call_summaries): New class. (call_sums): New variable. (ipa_profile_generate_summary): Generate indirect multiple targets summaries. (ipa_profile_write_edge_summary): New function. (ipa_profile_write_summary): Stream out indirect multiple targets summaries. (ipa_profile_dump_all_summaries): New function. (ipa_profile_read_edge_summary): New function. (ipa_profile_read_summary_section): New function. (ipa_profile_read_summary): Stream in indirect multiple targets summaries. (ipa_profile): Generate num_speculative_call_targets from profile summaries. * ipa-ref.h (speculative_id): New variable. * ipa-utils.c (ipa_merge_profiles): Update with target_prob. * lto-cgraph.c (lto_output_edge): Remove indirect common_target_id and common_target_probability. Stream out speculative_id and num_speculative_call_targets. (input_edge): Likewise. * predict.c (dump_prediction): Remove edges count assert to be precise. * symtab.c (symtab_node::create_reference): Init speculative_id. (symtab_node::clone_references): Clone speculative_id. (symtab_node::clone_referring): Clone speculative_id. (symtab_node::clone_reference): Clone speculative_id. (symtab_node::clear_stmts_in_references): Clear speculative_id. * tree-inline.c (copy_bb): Duplicate all the speculative edges if indirect call contains multiple speculative targets. * value-prof.h (check_ic_target): Remove. * value-prof.c (gimple_value_profile_transformations): Use void function gimple_ic_transform. * value-prof.c (gimple_ic_transform): Handle topn case. Fix comment typos. Change it to a void function. gcc/testsuite/ChangeLog 2020-01-14 Xiong Hu Luo PR ipa/69678 * gcc.dg/tree-prof/indir-call-prof-topn.c: New testcase. * gcc.dg/tree-prof/crossmodule-indir-call-topn-1.c: New testcase. * gcc.dg/tree-prof/crossmodule-indir-call-topn-1a.c: New testcase. * gcc.dg/tree-prof/crossmodule-indir-call-topn-2.c: New testcase. * lib/scandump.exp: Dump executable file name. * lib/scanwpaipa.exp: New scan-pgo-wap-ipa-dump. --- gcc/ipa-utils.c | 173 ++++++++++++++++++++++++++++++++++++++------------------ 1 file changed, 119 insertions(+), 54 deletions(-) (limited to 'gcc/ipa-utils.c') diff --git a/gcc/ipa-utils.c b/gcc/ipa-utils.c index 2e82980..587ad5c 100644 --- a/gcc/ipa-utils.c +++ b/gcc/ipa-utils.c @@ -675,68 +675,133 @@ ipa_merge_profiles (struct cgraph_node *dst, e2 = (e2 ? e2->next_callee : NULL), e = e->next_callee) { profile_count count = gimple_bb (e->call_stmt)->count; + /* Below code are introduced by r279373 of "Fix merging of common + traget info.". + + ipa-icf runs after ipa-profile, common_target_id with + common_target_probablity are useless in ipa-icf since they are + moved from cgraph.h to ipa-profile.c and processed already. + Need double circulation to find out each mapped direct speculative + edge and do prob merge. Not easy to construct a case to cover all + circumstances here. For src and dst both have multiple speculative + targets, only N:N maps are implemented, 2:0, 2:1, 0:2, 1:2 are not + implemented yet as too complicated and no test cases to cover. */ if (copy_counts) { - e->indirect_info->common_target_id - = e2->indirect_info->common_target_id; - e->indirect_info->common_target_probability - = e2->indirect_info->common_target_probability; + /* copy if both e and e2 have same num_speculative_call_targets. + */ + if (e->num_speculative_call_targets_p () + == e2->num_speculative_call_targets_p ()) + { + int num_specs = e->num_speculative_call_targets_p (); + cgraph_edge *direct, *indirect, *next_direct; + cgraph_edge *direct2, *indirect2, *next_direct2; + ipa_ref *ref; + for (next_direct = e; next_direct && num_specs--; + next_direct = direct->next_callee) + { + next_direct->speculative_call_info (direct, indirect, + ref); + + int num_specs2 = e2->num_speculative_call_targets_p (); + for (next_direct2 = e2; next_direct2 && num_specs2--; + next_direct2 = direct2->next_callee) + { + if (e2 && e2->speculative) + next_direct2->speculative_call_info (direct2, + indirect2, + ref); + if (direct->speculative_id == direct2->speculative_id + && direct->lto_stmt_uid == direct2->lto_stmt_uid) + { + direct->target_prob = direct2->target_prob; + break; + } + } + } + } + else + gcc_assert (e->num_speculative_call_targets_p () + && e->num_speculative_call_targets_p ()); } - else if (e->indirect_info->common_target_id - || e2->indirect_info->common_target_id) + else if (e->num_speculative_call_targets_p () + || e2->num_speculative_call_targets_p ()) { - sreal scale1 - = e->count.ipa().to_sreal_scale (count); - sreal scale2 - = e2->count.ipa().to_sreal_scale (count); - - if (scale1 == 0 && scale2 == 0) - scale1 = scale2 = 1; - sreal sum = scale1 + scale2; - int scaled_probability1 - = ((sreal)e->indirect_info->common_target_probability - * scale1 / sum).to_int (); - int scaled_probability2 - = ((sreal)e2->indirect_info->common_target_probability - * scale2 / sum).to_int (); - if (symtab->dump_file) + if (e->num_speculative_call_targets_p () + == e2->num_speculative_call_targets_p ()) { - fprintf (symtab->dump_file, - "Merging common targets %i prob %i" - " and %i prob %i with scales %f %f\n", - e->indirect_info->common_target_id, - e->indirect_info->common_target_probability, - e2->indirect_info->common_target_id, - e2->indirect_info->common_target_probability, - scale1.to_double (), - scale2.to_double ()); - fprintf (symtab->dump_file, "Combined BB count "); - count.dump (symtab->dump_file); - fprintf (symtab->dump_file, " dst edge count "); - e->count.dump (symtab->dump_file); - fprintf (symtab->dump_file, " src edge count "); - e2->count.dump (symtab->dump_file); - fprintf (symtab->dump_file, "\n"); + int num_specs = e->num_speculative_call_targets_p (); + cgraph_edge *direct, *indirect, *next_direct; + cgraph_edge *direct2, *indirect2, *next_direct2; + ipa_ref *ref; + for (next_direct = e; next_direct && num_specs--; + next_direct = direct->next_callee) + { + next_direct->speculative_call_info (direct, indirect, + ref); + + int num_specs2 = e2->num_speculative_call_targets_p (); + for (next_direct2 = e2; next_direct2 && num_specs2--; + next_direct2 = direct2->next_callee) + { + if (e2 && e2->speculative) + next_direct2->speculative_call_info (direct2, + indirect2, + ref); + if (direct->speculative_id == direct2->speculative_id + && direct->lto_stmt_uid == direct2->lto_stmt_uid) + { + sreal scale1 + = e->count.ipa ().to_sreal_scale (count); + sreal scale2 + = e2->count.ipa ().to_sreal_scale (count); + + if (scale1 == 0 && scale2 == 0) + scale1 = scale2 = 1; + sreal sum = scale1 + scale2; + int scaled_prob1 + = (((sreal)direct->target_prob) + * scale1 / sum).to_int (); + int scaled_prob2 + = (((sreal)direct2->target_prob) + * scale2 / sum).to_int (); + if (symtab->dump_file) + { + fprintf ( + symtab->dump_file, + "Merging speculative id %i prob %i" + " and %i prob %i with scales %f %f\n", + direct->speculative_id, direct->target_prob, + direct2->speculative_id, + direct2->target_prob, scale1.to_double (), + scale2.to_double ()); + fprintf (symtab->dump_file, + "Combined BB count "); + count.dump (symtab->dump_file); + fprintf (symtab->dump_file, + " dst edge count "); + e->count.dump (symtab->dump_file); + fprintf (symtab->dump_file, + " src edge count "); + e2->count.dump (symtab->dump_file); + fprintf (symtab->dump_file, "\n"); + } + direct->target_prob = scaled_prob1 + scaled_prob2; + break; + } + } + } } - if (e->indirect_info->common_target_id - == e2->indirect_info->common_target_id) - e->indirect_info->common_target_probability - = scaled_probability1 + scaled_probability2; - else if (!e2->indirect_info->common_target_id - || scaled_probability1 > scaled_probability2) - e->indirect_info->common_target_probability - = scaled_probability1; - else + else if (e->num_speculative_call_targets_p ()) { - e->indirect_info->common_target_id - = e2->indirect_info->common_target_id; - e->indirect_info->common_target_probability - = scaled_probability2; + /* Process if only dst is speculative. */ + gcc_assert (!e->num_speculative_call_targets_p ()); + } + else if (e2->num_speculative_call_targets_p ()) + { + /* Process if only src is speculative. */ + gcc_assert (!e2->num_speculative_call_targets_p ()); } - if (symtab->dump_file) - fprintf (symtab->dump_file, "Merged as %i prob %i\n", - e->indirect_info->common_target_id, - e->indirect_info->common_target_probability); } /* When call is speculative, we need to re-distribute probabilities -- cgit v1.1 From 845bb366adcf702331de3d8022fd0e1c1c918607 Mon Sep 17 00:00:00 2001 From: Jan Hubicka Date: Tue, 28 Jan 2020 20:34:56 +0100 Subject: ipa: fix handling of multiple speculations (PR93318) This patch started as work to resole Richard's comment on quadratic lookups in resolve_speculation. While doing it I however noticed multiple problems in the new speuclative call code which made the patch quite big. In particular: 1) Before applying speculation we consider only targets with at lest probability 1/2. If profile is sane at most two targets can have probability greater or equal to 1/2. So the new multi-target speculation code got enabled only in very special scenario when there ae precisely two target with precise probability 1/2 (which is tested by the single testcase). As a conseuqence the multiple target logic got minimal test coverage and this made us to miss several ICEs. 2) Profile updating in profile merging, tree-inline and indirect call expansion was wrong which led to inconsistent profiles (as already seen on the testcase). 3) Code responsible to turn speculative call to direct call was broken for anything with more than one target. 4) There were multiple cases where call_site_hash went out of sync which eventually leads to an ICE.. 5) Some code expects that all speculative call targets forms a sequence in the callee linked list but there is no code to maintain that invariant nor a verifier. Fixing this it became obvious that the current API of speculative_call_info is not useful because it really builds on fact tht there are precisely three components (direct call, ref and indirect call) in every speculative call sequence. I ended up replacing it with iterator API for direct call (first_speculative_call_target, next_speculative_call_target) and accessors for the other coponents updating comment in cgraph.h. Finally I made the work with call site hash more effetive by updating edge manipulation to keep them in sequence. So first one can be looked up from the hash and then they can be iterated by callee. There are other things that can be improved (for example the speculation should start with most common target first), but I will try to keep that for next stage1. This patch is mostly about getting rid of ICE and profile corruption which is a regression from GCC 9. gcc/ChangeLog: PR lto/93318 * cgraph.c (cgraph_add_edge_to_call_site_hash): Update call site hash only when edge is first within the sequence. (cgraph_edge::set_call_stmt): Update handling of speculative calls. (symbol_table::create_edge): Do not set target_prob. (cgraph_edge::remove_caller): Watch for speculative calls when updating the call site hash. (cgraph_edge::make_speculative): Drop target_prob parameter. (cgraph_edge::speculative_call_info): Remove. (cgraph_edge::first_speculative_call_target): New member function. (update_call_stmt_hash_for_removing_direct_edge): New function. (cgraph_edge::resolve_speculation): Rewrite to new API. (cgraph_edge::speculative_call_for_target): New member function. (cgraph_edge::make_direct): Rewrite to new API; fix handling of multiple speculation targets. (cgraph_edge::redirect_call_stmt_to_callee): Likewise; fix updating of profile. (verify_speculative_call): Verify that targets form an interval. * cgraph.h (cgraph_edge::speculative_call_info): Remove. (cgraph_edge::first_speculative_call_target): New member function. (cgraph_edge::next_speculative_call_target): New member function. (cgraph_edge::speculative_call_target_ref): New member function. (cgraph_edge;:speculative_call_indirect_edge): New member funtion. (cgraph_edge): Remove target_prob. * cgraphclones.c (cgraph_node::set_call_stmt_including_clones): Fix handling of speculative calls. * ipa-devirt.c (ipa_devirt): Fix handling of speculative cals. * ipa-fnsummary.c (analyze_function_body): Likewise. * ipa-inline.c (speculation_useful_p): Use new speculative call API. * ipa-profile.c (dump_histogram): Fix formating. (ipa_profile_generate_summary): Watch for overflows. (ipa_profile): Do not require probablity to be 1/2; update to new API. * ipa-prop.c (ipa_make_edge_direct_to_target): Update to new API. (update_indirect_edges_after_inlining): Update to new API. * ipa-utils.c (ipa_merge_profiles): Rewrite merging of speculative call profiles. * profile-count.h: (profile_probability::adjusted): New. * tree-inline.c (copy_bb): Update to new speculative call API; fix updating of profile. * value-prof.c (gimple_ic_transform): Rename to ... (dump_ic_profile): ... this one; update dumping. (stream_in_histogram_value): Fix formating. (gimple_value_profile_transformations): Update. gcc/testsuite/ChangeLog: * g++.dg/tree-prof/indir-call-prof.C: Update template. * gcc.dg/tree-prof/crossmodule-indircall-1.c: Add more targets. * gcc.dg/tree-prof/crossmodule-indircall-1a.c: Add more targets. * gcc.dg/tree-prof/indir-call-prof.c: Update template. --- gcc/ipa-utils.c | 241 ++++++++++++++------------------------------------------ 1 file changed, 58 insertions(+), 183 deletions(-) (limited to 'gcc/ipa-utils.c') diff --git a/gcc/ipa-utils.c b/gcc/ipa-utils.c index 587ad5c..23e7f71 100644 --- a/gcc/ipa-utils.c +++ b/gcc/ipa-utils.c @@ -674,201 +674,75 @@ ipa_merge_profiles (struct cgraph_node *dst, for (e = dst->indirect_calls, e2 = src->indirect_calls; e; e2 = (e2 ? e2->next_callee : NULL), e = e->next_callee) { - profile_count count = gimple_bb (e->call_stmt)->count; - /* Below code are introduced by r279373 of "Fix merging of common - traget info.". - - ipa-icf runs after ipa-profile, common_target_id with - common_target_probablity are useless in ipa-icf since they are - moved from cgraph.h to ipa-profile.c and processed already. - Need double circulation to find out each mapped direct speculative - edge and do prob merge. Not easy to construct a case to cover all - circumstances here. For src and dst both have multiple speculative - targets, only N:N maps are implemented, 2:0, 2:1, 0:2, 1:2 are not - implemented yet as too complicated and no test cases to cover. */ - if (copy_counts) + if (!e->speculative && !e2->speculative) { - /* copy if both e and e2 have same num_speculative_call_targets. - */ - if (e->num_speculative_call_targets_p () - == e2->num_speculative_call_targets_p ()) - { - int num_specs = e->num_speculative_call_targets_p (); - cgraph_edge *direct, *indirect, *next_direct; - cgraph_edge *direct2, *indirect2, *next_direct2; - ipa_ref *ref; - for (next_direct = e; next_direct && num_specs--; - next_direct = direct->next_callee) - { - next_direct->speculative_call_info (direct, indirect, - ref); - - int num_specs2 = e2->num_speculative_call_targets_p (); - for (next_direct2 = e2; next_direct2 && num_specs2--; - next_direct2 = direct2->next_callee) - { - if (e2 && e2->speculative) - next_direct2->speculative_call_info (direct2, - indirect2, - ref); - if (direct->speculative_id == direct2->speculative_id - && direct->lto_stmt_uid == direct2->lto_stmt_uid) - { - direct->target_prob = direct2->target_prob; - break; - } - } - } - } - else - gcc_assert (e->num_speculative_call_targets_p () - && e->num_speculative_call_targets_p ()); + /* FIXME: we need to also merge ipa-profile histograms + because with LTO merging happens from lto-symtab before + these are converted to indirect edges. */ + e->count = gimple_bb (e->call_stmt)->count; + continue; } - else if (e->num_speculative_call_targets_p () - || e2->num_speculative_call_targets_p ()) + + /* When copying just remove all speuclations on dst and then copy + one from src. */ + if (copy_counts) { - if (e->num_speculative_call_targets_p () - == e2->num_speculative_call_targets_p ()) + while (e->speculative) + cgraph_edge::resolve_speculation (e, NULL); + e->count = gimple_bb (e->call_stmt)->count; + if (e2->speculative) { - int num_specs = e->num_speculative_call_targets_p (); - cgraph_edge *direct, *indirect, *next_direct; - cgraph_edge *direct2, *indirect2, *next_direct2; - ipa_ref *ref; - for (next_direct = e; next_direct && num_specs--; - next_direct = direct->next_callee) + for (cgraph_edge *e3 = e2->first_speculative_call_target (); + e3; + e3 = e3->next_speculative_call_target ()) { - next_direct->speculative_call_info (direct, indirect, - ref); - - int num_specs2 = e2->num_speculative_call_targets_p (); - for (next_direct2 = e2; next_direct2 && num_specs2--; - next_direct2 = direct2->next_callee) - { - if (e2 && e2->speculative) - next_direct2->speculative_call_info (direct2, - indirect2, - ref); - if (direct->speculative_id == direct2->speculative_id - && direct->lto_stmt_uid == direct2->lto_stmt_uid) - { - sreal scale1 - = e->count.ipa ().to_sreal_scale (count); - sreal scale2 - = e2->count.ipa ().to_sreal_scale (count); - - if (scale1 == 0 && scale2 == 0) - scale1 = scale2 = 1; - sreal sum = scale1 + scale2; - int scaled_prob1 - = (((sreal)direct->target_prob) - * scale1 / sum).to_int (); - int scaled_prob2 - = (((sreal)direct2->target_prob) - * scale2 / sum).to_int (); - if (symtab->dump_file) - { - fprintf ( - symtab->dump_file, - "Merging speculative id %i prob %i" - " and %i prob %i with scales %f %f\n", - direct->speculative_id, direct->target_prob, - direct2->speculative_id, - direct2->target_prob, scale1.to_double (), - scale2.to_double ()); - fprintf (symtab->dump_file, - "Combined BB count "); - count.dump (symtab->dump_file); - fprintf (symtab->dump_file, - " dst edge count "); - e->count.dump (symtab->dump_file); - fprintf (symtab->dump_file, - " src edge count "); - e2->count.dump (symtab->dump_file); - fprintf (symtab->dump_file, "\n"); - } - direct->target_prob = scaled_prob1 + scaled_prob2; - break; - } - } + cgraph_edge *ns; + ns = e->make_speculative + (dyn_cast + (e3->speculative_call_target_ref ()->referred), + e3->count, e3->speculative_id); + /* Target may differ from ref (for example it may be + redirected to local alias. */ + ns->redirect_callee (e3->callee); } } - else if (e->num_speculative_call_targets_p ()) - { - /* Process if only dst is speculative. */ - gcc_assert (!e->num_speculative_call_targets_p ()); - } - else if (e2->num_speculative_call_targets_p ()) - { - /* Process if only src is speculative. */ - gcc_assert (!e2->num_speculative_call_targets_p ()); - } + continue; } - /* When call is speculative, we need to re-distribute probabilities - the same way as they was. This is not really correct because - in the other copy the speculation may differ; but probably it - is not really worth the effort. */ - if (e->speculative) + /* Iterate all speculations in SRC, see if corresponding ones exist + int DST and if so, sum the counts. Otherwise create new + speculation. */ + int max_spec = 0; + for (cgraph_edge *e3 = e->first_speculative_call_target (); + e3; + e3 = e3->next_speculative_call_target ()) + if (e3->speculative_id > max_spec) + max_spec = e3->speculative_id; + for (cgraph_edge *e3 = e2->first_speculative_call_target (); + e3; + e3 = e3->next_speculative_call_target ()) { - cgraph_edge *direct, *indirect; - cgraph_edge *direct2 = NULL, *indirect2 = NULL; - ipa_ref *ref; - - e->speculative_call_info (direct, indirect, ref); - gcc_assert (e == indirect); - if (e2 && e2->speculative) - e2->speculative_call_info (direct2, indirect2, ref); - if (indirect->count > profile_count::zero () - || direct->count > profile_count::zero ()) + cgraph_edge *te + = e->speculative_call_for_target + (dyn_cast + (e3->speculative_call_target_ref ()->referred)); + if (te) + te->count = te->count + e3->count; + else { - /* We should mismatch earlier if there is no matching - indirect edge. */ - if (!e2) - { - if (symtab->dump_file) - fprintf (symtab->dump_file, - "Mismatch in merging indirect edges\n"); - } - else if (!e2->speculative) - indirect->count += e2->count; - else if (e2->speculative) - { - if (DECL_ASSEMBLER_NAME (direct2->callee->decl) - != DECL_ASSEMBLER_NAME (direct->callee->decl)) - { - if (direct2->count >= direct->count) - { - direct->redirect_callee (direct2->callee); - indirect->count += indirect2->count - + direct->count; - direct->count = direct2->count; - } - else - indirect->count += indirect2->count + direct2->count; - } - else - { - direct->count += direct2->count; - indirect->count += indirect2->count; - } - } + e->count = e->count + e3->count; + cgraph_edge *ns; + ns = e->make_speculative + (dyn_cast + (e3->speculative_call_target_ref () + ->referred), + e3->count, + e3->speculative_id + max_spec + 1); + /* Target may differ from ref (for example it may be + redirected to local alias. */ + ns->redirect_callee (e3->callee); } - else - /* At the moment we should have only profile feedback based - speculations when merging. */ - gcc_unreachable (); } - else if (e2 && e2->speculative) - { - cgraph_edge *direct, *indirect; - ipa_ref *ref; - - e2->speculative_call_info (direct, indirect, ref); - e->count = count; - e->make_speculative (direct->callee, direct->count); - } - else - e->count = count; } if (!preserve_body) src->release_body (); @@ -882,7 +756,8 @@ ipa_merge_profiles (struct cgraph_node *dst, src->decl = oldsrcdecl; } -/* Return true if call to DEST is known to be self-recusive call withing FUNC. */ +/* Return true if call to DEST is known to be self-recusive + call withing FUNC. */ bool recursive_call_p (tree func, tree dest) -- cgit v1.1