aboutsummaryrefslogtreecommitdiff
path: root/gcc/tree-vect-loop-manip.c
AgeCommit message (Collapse)AuthorFilesLines
2020-12-07tree-optimization/98117 - fix range set by vectorization on niter IVsRichard Biener1-6/+22
This avoids the degenerate case of a TYPE_MAX_VALUE latch iteration count value causing wrong range info for the vector IV. There's still the case of VF == 1 where if we don't know whether we hit the above case we cannot emit a range. 2020-12-07 Richard Biener <rguenther@suse.de> PR tree-optimization/98117 * tree-vect-loop-manip.c (vect_gen_vector_loop_niters): Properly handle degenerate niter when setting the vector loop IV range. * gcc.dg/torture/pr98117.c: New testcase.
2020-10-27SLP vectorize across PHI nodesRichard Biener1-0/+27
This makes SLP discovery detect backedges by seeding the bst_map with the node to be analyzed so it can be picked up from recursive calls. This removes the need to discover backedges in a separate walk. This enables SLP build to handle PHI nodes in full, continuing the SLP build to non-backedges. For loop vectorization this enables outer loop vectorization of nested SLP cycles and for BB vectorization this enables vectorization of PHIs at CFG merges. It also turns code generation into a SCC discovery walk to handle irreducible regions and nodes only reachable via backedges where we now also fill in vectorized backedge defs. This requires sanitizing the SLP tree for SLP reduction chains even more, manually filling the backedge SLP def. This also exposes the fact that CFG copying (and edge splitting until I fixed that) ends up with different edge order in the copy which doesn't play well with the desired 1:1 mapping of SLP PHI node children and edges for epilogue vectorization. I've tried to fixup CFG copying here but this really looks like a dead (or expensive) end there so I've done fixup in slpeel_tree_duplicate_loop_to_edge_cfg instead for the cases we can run into. There's still NULLs in the SLP_TREE_CHILDREN vectors and I'm not sure it's possible to eliminate them all this stage1 so the patch has quite some checks for this case all over the place. Bootstrapped and tested on x86_64-unknown-linux-gnu. SPEC CPU 2017 and SPEC CPU 2006 successfully built and tested. 2020-10-27 Richard Biener <rguenther@suse.de> * gimple.h (gimple_expr_type): For PHIs return the type of the result. * tree-vect-loop-manip.c (slpeel_tree_duplicate_loop_to_edge_cfg): Make sure edge order into copied loop headers line up with the originals. * tree-vect-loop.c (vect_transform_cycle_phi): Handle nested loops with SLP. (vectorizable_phi): New function. (vectorizable_live_operation): For BB vectorization compute insert location here. * tree-vect-slp.c (vect_free_slp_tree): Deal with NULL SLP_TREE_CHILDREN entries. (vect_create_new_slp_node): Add overloads with pre-existing node argument. (vect_print_slp_graph): Likewise. (vect_mark_slp_stmts): Likewise. (vect_mark_slp_stmts_relevant): Likewise. (vect_gather_slp_loads): Likewise. (vect_optimize_slp): Likewise. (vect_slp_analyze_node_operations): Likewise. (vect_bb_slp_scalar_cost): Likewise. (vect_remove_slp_scalar_calls): Likewise. (vect_get_and_check_slp_defs): Handle PHIs. (vect_build_slp_tree_1): Handle PHIs. (vect_build_slp_tree_2): Continue SLP build, following PHI arguments. Fix memory leak. (vect_build_slp_tree): Put stub node into the hash-map so we can discover cycles directly. (vect_build_slp_instance): Set the backedge SLP def for reduction chains. (vect_analyze_slp_backedges): Remove. (vect_analyze_slp): Do not call it. (vect_slp_convert_to_external): Release SLP_TREE_LOAD_PERMUTATION. (vect_slp_analyze_node_operations): Handle stray failed backedge defs by failing. (vect_slp_build_vertices): Adjust leaf condition. (vect_bb_slp_mark_live_stmts): Handle PHIs, use visited hash-set to handle cycles. (vect_slp_analyze_operations): Adjust. (vect_bb_partition_graph_r): Likewise. (vect_slp_function): Adjust split condition to allow CFG merges. (vect_schedule_slp_instance): Rename to ... (vect_schedule_slp_node): ... this. Move DFS walk to ... (vect_schedule_scc): ... this new function. (vect_schedule_slp): Call it. Remove ad-hoc vectorized backedge fill code. * tree-vect-stmts.c (vect_analyze_stmt): Call vectorizable_phi. (vect_transform_stmt): Likewise. (vect_is_simple_use): Handle vect_backedge_def. * tree-vectorizer.c (vec_info::new_stmt_vec_info): Only set loop header PHIs to vect_unknown_def_type for loop vectorization. * tree-vectorizer.h (enum vect_def_type): Add vect_backedge_def. (enum stmt_vec_info_type): Add phi_info_type. (vectorizable_phi): Declare. * gcc.dg/vect/bb-slp-54.c: New test. * gcc.dg/vect/bb-slp-55.c: Likewise. * gcc.dg/vect/bb-slp-56.c: Likewise. * gcc.dg/vect/bb-slp-57.c: Likewise. * gcc.dg/vect/bb-slp-58.c: Likewise. * gcc.dg/vect/bb-slp-59.c: Likewise. * gcc.dg/vect/bb-slp-60.c: Likewise. * gcc.dg/vect/bb-slp-61.c: Likewise. * gcc.dg/vect/bb-slp-62.c: Likewise. * gcc.dg/vect/bb-slp-63.c: Likewise. * gcc.dg/vect/bb-slp-64.c: Likewise. * gcc.dg/vect/bb-slp-65.c: Likewise. * gcc.dg/vect/bb-slp-66.c: Likewise. * gcc.dg/vect/vect-outer-slp-1.c: Likewise. * gfortran.dg/vect/O3-bb-slp-1.f: Likewise. * gfortran.dg/vect/O3-bb-slp-2.f: Likewise. * g++.dg/vect/simd-11.cc: Likewise.
2020-10-26tree-optimization/97539 - reset out-of-loop debug uses before peelingRichard Biener1-1/+40
This makes sure to reset out-of-loop debug uses before vectorizer loop peeling as we cannot make sure to retain the use-def dominance relationship when there are no LC SSA nodes. 2020-10-26 Richard Biener <rguenther@suse.de> PR tree-optimization/97539 * tree-vect-loop-manip.c (vect_do_peeling): Reset out-of-loop debug uses before peeling. * gcc.dg/pr97539.c: New testcase.
2020-09-23vect: Fix epilogue loop handling of partial vectorsRichard Sandiford1-26/+57
This patch fixes the fallout that Kewen reported on Power after the recent change to avoid unnecessary use of partial vectors. As Kewen said, the problem is that vect_analyze_loop_2 doesn't know how many epilogue iterations there will be, and so it cannot make a final decision about whether the number of iterations forces an epilogue loop to use partial vectors. This is similar to the current situation for peeling: we don't know during initial analysis whether an epilogue loop will itself require peeling. Instead we decide that during vect_do_peeling, where the final number of epilogue loop iterations is known. The patch takes a similar approach for the decision about whether to use partial vectors. As the comments in the patch say, the idea is that vect_analyze_loop_2 should make peeling and partial- vector decisions based on the assumption that the loop_vinfo will be used as the main loop, while vect_do_peeling should make them in the knowledge that the loop_vinfo will be used as an epilogue loop. This allows the same analysis to be used for both cases, which we rely on for implementing VECT_COMPARE_COSTS; see the big comment in vect_analyze_loop for details. I hope the patch makes the (mostly preexisting) structure a bit more obvious. It isn't what anyone would design from scratch, but that's the nature of working with a mature vector framework. Arranging things this way means that vect_verify_full_masking and vect_verify_loop_lens now become part of the “can” rather than “will” test for partial vectors. Also, while splitting out the logic that handles epilogues with constant iterations, I added a check to make sure that we don't try to use partial vectors to vectorise a single-scalar loop. This required some changes to the Power tests. gcc/ * tree-vectorizer.h (determine_peel_for_niter): Delete in favor of... (vect_determine_partial_vectors_and_peeling): ...this new function. * tree-vect-loop-manip.c (vect_update_epilogue_niters): New function. Reject using vector epilogue loops for single iterations. Install the constant number of epilogue loop iterations in the associated loop_vinfo. Rely on vect_determine_partial_vectors_and_peeling to do the main part of the test. (vect_do_peeling): Use vect_update_epilogue_niters to handle epilogue loops with a known number of iterations. Skip recomputing the number of iterations later in that case. Otherwise, use vect_determine_partial_vectors_and_peeling to decide whether the epilogue loop needs to use partial vectors or peeling. * tree-vect-loop.c (_loop_vec_info::_loop_vec_info): Set the default can_use_partial_vectors_p to false if partial-vector-usage=0. (determine_peel_for_niter): Remove in favor of... (vect_determine_partial_vectors_and_peeling): ...this new function, split out from... (vect_analyze_loop_2): ...here. Reflect the vect_verify_full_masking and vect_verify_loop_lens results in CAN_USE_PARTIAL_VECTORS_P rather than USING_PARTIAL_VECTORS_P. gcc/testsuite/ * gcc.target/powerpc/p9-vec-length-epil-1.c: Do not expect the single-iteration epilogues of the 64-bit loops to be vectorized. * gcc.target/powerpc/p9-vec-length-epil-7.c: Likewise. * gcc.target/powerpc/p9-vec-length-epil-8.c: Likewise.
2020-08-06vect/rs6000: Support vector with length cost modelingKewen Lin1-8/+6
This patch is to add the cost modeling for vector with length, it mainly follows what we generate for vector with length in functions vect_set_loop_controls_directly and vect_gen_len at the worst case. For Power, the length is expected to be in bits 0-7 (high bits), we have to model the cost of shifting bits, which is implemented in adjust_vect_cost_per_loop. Bootstrapped/regtested on powerpc64le-linux-gnu (P9) with explicit param vect-partial-vector-usage=1. gcc/ChangeLog: * config/rs6000/rs6000.c (rs6000_adjust_vect_cost_per_loop): New function. (rs6000_finish_cost): Call rs6000_adjust_vect_cost_per_loop. * tree-vect-loop.c (vect_estimate_min_profitable_iters): Add cost modeling for vector with length. (vect_rgroup_iv_might_wrap_p): New function, factored out from... * tree-vect-loop-manip.c (vect_set_loop_controls_directly): ...this. Update function comment. * tree-vect-stmts.c (vect_gen_len): Update function comment. * tree-vectorizer.h (vect_rgroup_iv_might_wrap_p): New declare.
2020-07-19vect: Support length-based partial vectors approachKewen Lin1-73/+109
Power9 supports vector load/store instruction lxvl/stxvl which allow us to operate partial vectors with one specific length. This patch extends some of current mask-based partial vectors support code for length-based approach, also adds some length specific support code. So far it assumes that we can only have one partial vectors approach at the same time, it will disable to use partial vectors if both approaches co-exist. Like the description of optab len_load/len_store, the length-based approach can have two flavors, one is length in bytes, the other is length in lanes. This patch is mainly implemented and tested for length in bytes, but as Richard S. suggested, most of code has considered both flavors. This also introduces one parameter vect-partial-vector-usage allow users to control when the loop vectorizer considers using partial vectors as an alternative to falling back to scalar code. gcc/ChangeLog: * config/rs6000/rs6000.c (rs6000_option_override_internal): Set param_vect_partial_vector_usage to 0 explicitly. * doc/invoke.texi (vect-partial-vector-usage): Document new option. * optabs-query.c (get_len_load_store_mode): New function. * optabs-query.h (get_len_load_store_mode): New declare. * params.opt (vect-partial-vector-usage): New. * tree-vect-loop-manip.c (vect_set_loop_controls_directly): Add the handlings for vectorization using length-based partial vectors, call vect_gen_len for length generation, and rename some variables with items instead of scalars. (vect_set_loop_condition_partial_vectors): Add the handlings for vectorization using length-based partial vectors. (vect_do_peeling): Allow remaining eiters less than epilogue vf for LOOP_VINFO_USING_PARTIAL_VECTORS_P. * tree-vect-loop.c (_loop_vec_info::_loop_vec_info): Init epil_using_partial_vectors_p. (_loop_vec_info::~_loop_vec_info): Call release_vec_loop_controls for lengths destruction. (vect_verify_loop_lens): New function. (vect_analyze_loop): Add handlings for epilogue of loop when it's marked to use vectorization using partial vectors. (vect_analyze_loop_2): Add the check to allow only one vectorization approach using partial vectorization at the same time. Check param vect-partial-vector-usage for partial vectors decision. Mark LOOP_VINFO_EPIL_USING_PARTIAL_VECTORS_P if the epilogue is considerable to use partial vectors. Call release_vec_loop_controls for lengths destruction. (vect_estimate_min_profitable_iters): Adjust for loop vectorization using length-based partial vectors. (vect_record_loop_mask): Init factor to 1 for vectorization using mask-based partial vectors. (vect_record_loop_len): New function. (vect_get_loop_len): Likewise. * tree-vect-stmts.c (check_load_store_for_partial_vectors): Add checks for vectorization using length-based partial vectors. Factor some code to lambda function get_valid_nvectors. (vectorizable_store): Add handlings when using length-based partial vectors. (vectorizable_load): Likewise. (vect_gen_len): New function. * tree-vectorizer.h (struct rgroup_controls): Add field factor mainly for length-based partial vectors. (vec_loop_lens): New typedef. (_loop_vec_info): Add lens and epil_using_partial_vectors_p. (LOOP_VINFO_EPIL_USING_PARTIAL_VECTORS_P): New macro. (LOOP_VINFO_LENS): Likewise. (LOOP_VINFO_FULLY_WITH_LENGTH_P): Likewise. (vect_record_loop_len): New declare. (vect_get_loop_len): Likewise. (vect_gen_len): Likewise.
2020-06-17tree-optimization/95717 - fix SSA update for vectorizer epilogueRichard Biener1-4/+4
This fixes yet another issue with the custom SSA updating in the vectorizer when we copy from the non-if-converted loop. We must not mess with current defs before we updated the BB copies. 2020-06-17 Richard Biener <rguenther@suse.de> PR tree-optimization/95717 * tree-vect-loop-manip.c (slpeel_tree_duplicate_loop_to_edge_cfg): Move BB SSA updating before exit/latch PHI current def copying. * g++.dg/torture/pr95717.C: New testcase.
2020-06-12vect: Factor out and rename some functions/macrosKewen Lin1-21/+21
Power supports vector memory access with length (in bytes) instructions. Like existing fully masking for SVE, it is another approach to vectorize the loop using partially-populated vectors. As Richard Sandiford suggested, we should share the codes in approaches with partial vectors if possible. This patch is to: 1) factor out two functions: - vect_min_prec_for_max_niters - vect_known_niters_smaller_than_vf. 2) rename four functions: - vect_iv_limit_for_full_masking - check_load_store_masking - vect_set_loop_condition_masked - vect_set_loop_condition_unmasked 3) rename macros LOOP_VINFO_MASK_COMPARE_TYPE and LOOP_VINFO_MASK_IV_TYPE. Bootstrapped/regtested on aarch64-linux-gnu. gcc/ChangeLog: * tree-vect-loop-manip.c (vect_set_loop_controls_directly): Rename LOOP_VINFO_MASK_COMPARE_TYPE to LOOP_VINFO_RGROUP_COMPARE_TYPE. Rename LOOP_VINFO_MASK_IV_TYPE to LOOP_VINFO_RGROUP_IV_TYPE. (vect_set_loop_condition_masked): Renamed to ... (vect_set_loop_condition_partial_vectors): ... this. Rename LOOP_VINFO_MASK_COMPARE_TYPE to LOOP_VINFO_RGROUP_COMPARE_TYPE. Rename vect_iv_limit_for_full_masking to vect_iv_limit_for_partial_vectors. (vect_set_loop_condition_unmasked): Renamed to ... (vect_set_loop_condition_normal): ... this. (vect_set_loop_condition): Rename vect_set_loop_condition_unmasked to vect_set_loop_condition_normal. Rename vect_set_loop_condition_masked to vect_set_loop_condition_partial_vectors. (vect_prepare_for_masked_peels): Rename LOOP_VINFO_MASK_COMPARE_TYPE to LOOP_VINFO_RGROUP_COMPARE_TYPE. * tree-vect-loop.c (vect_known_niters_smaller_than_vf): New, factored out from ... (vect_analyze_loop_costing): ... this. (_loop_vec_info::_loop_vec_info): Rename mask_compare_type to compare_type. (vect_min_prec_for_max_niters): New, factored out from ... (vect_verify_full_masking): ... this. Rename vect_iv_limit_for_full_masking to vect_iv_limit_for_partial_vectors. Rename LOOP_VINFO_MASK_COMPARE_TYPE to LOOP_VINFO_RGROUP_COMPARE_TYPE. Rename LOOP_VINFO_MASK_IV_TYPE to LOOP_VINFO_RGROUP_IV_TYPE. (vectorizable_reduction): Update some dumpings with partial vectors instead of fully-masked. (vectorizable_live_operation): Likewise. (vect_iv_limit_for_full_masking): Renamed to ... (vect_iv_limit_for_partial_vectors): ... this. * tree-vect-stmts.c (check_load_store_masking): Renamed to ... (check_load_store_for_partial_vectors): ... this. Update some dumpings with partial vectors instead of fully-masked. (vectorizable_store): Rename check_load_store_masking to check_load_store_for_partial_vectors. (vectorizable_load): Likewise. * tree-vectorizer.h (LOOP_VINFO_MASK_COMPARE_TYPE): Renamed to ... (LOOP_VINFO_RGROUP_COMPARE_TYPE): ... this. (LOOP_VINFO_MASK_IV_TYPE): Renamed to ... (LOOP_VINFO_RGROUP_IV_TYPE): ... this. (vect_iv_limit_for_full_masking): Renamed to ... (vect_iv_limit_for_partial_vectors): this. (_loop_vec_info): Rename mask_compare_type to rgroup_compare_type. Rename iv_type to rgroup_iv_type.
2020-06-11vect: Rename things related to rgroup_masksKewen Lin1-85/+86
Power supports vector memory access with length (in bytes) instructions. Like existing fully masking for SVE, it is another approach to vectorize the loop using partially-populated vectors. As Richard Sandiford pointed out, we can rename the rgroup struct rgroup_masks to rgroup_controls, rename its members mask_type to type, masks to controls to be more generic. Besides, this patch also renames some functions like vect_set_loop_mask to vect_set_loop_control, release_vec_loop_masks to release_vec_loop_controls, vect_set_loop_masks_directly to vect_set_loop_controls_directly. Bootstrapped/regtested on aarch64-linux-gnu. gcc/ChangeLog: * tree-vect-loop-manip.c (vect_set_loop_mask): Renamed to ... (vect_set_loop_control): ... this. (vect_maybe_permute_loop_masks): Rename rgroup_masks related things. (vect_set_loop_masks_directly): Renamed to ... (vect_set_loop_controls_directly): ... this. Also rename some variables with ctrl instead of mask. Rename vect_set_loop_mask to vect_set_loop_control. (vect_set_loop_condition_masked): Rename rgroup_masks related things. Also rename some variables with ctrl instead of mask. * tree-vect-loop.c (release_vec_loop_masks): Renamed to ... (release_vec_loop_controls): ... this. Rename rgroup_masks related things. (_loop_vec_info::~_loop_vec_info): Rename release_vec_loop_masks to release_vec_loop_controls. (can_produce_all_loop_masks_p): Rename rgroup_masks related things. (vect_get_max_nscalars_per_iter): Likewise. (vect_estimate_min_profitable_iters): Likewise. (vect_record_loop_mask): Likewise. (vect_get_loop_mask): Likewise. * tree-vectorizer.h (struct rgroup_masks): Renamed to ... (struct rgroup_controls): ... this. Also rename mask_type to type and rename masks to controls.
2020-06-11vect: Rename fully_masked_p to using_partial_vectors_pKewen Lin1-3/+3
Power supports vector memory access with length (in bytes) instructions. Like existing fully masking for SVE, it is another approach to vectorize the loop using partially-populated vectors. As Richard Sandiford suggested, this patch is to update the existing fully_masked_p field to using_partial_vectors_p. Introduce one macro LOOP_VINFO_USING_PARTIAL_VECTORS_P for partial vectorization checking usage, update the LOOP_VINFO_FULLY_MASKED_P with LOOP_VINFO_USING_PARTIAL_VECTORS_P && !masks.is_empty() and still use it for mask-based partial vectors approach specific checks. Bootstrapped/regtested on aarch64-linux-gnu. gcc/ChangeLog: * tree-vect-loop-manip.c (vect_set_loop_condition): Rename LOOP_VINFO_FULLY_MASKED_P to LOOP_VINFO_USING_PARTIAL_VECTORS_P. (vect_gen_vector_loop_niters): Likewise. (vect_do_peeling): Likewise. * tree-vect-loop.c (_loop_vec_info::_loop_vec_info): Rename fully_masked_p to using_partial_vectors_p. (vect_analyze_loop_costing): Rename LOOP_VINFO_FULLY_MASKED_P to LOOP_VINFO_USING_PARTIAL_VECTORS_P. (determine_peel_for_niter): Likewise. (vect_estimate_min_profitable_iters): Likewise. (vect_transform_loop): Likewise. * tree-vectorizer.h (LOOP_VINFO_FULLY_MASKED_P): Updated. (LOOP_VINFO_USING_PARTIAL_VECTORS_P): New macro.
2020-05-05add vec_info * parameters where neededRichard Biener1-2/+4
Soonish we'll get SLP nodes which have no corresponding scalar stmt and thus not stmt_vec_info and thus no way to get back to the associated vec_info. This patch makes the vec_info available as part of the APIs instead of putting in that back-pointer into the leaf data structures. 2020-05-05 Richard Biener <rguenther@suse.de> * tree-vectorizer.h (_stmt_vec_info::vinfo): Remove. (STMT_VINFO_LOOP_VINFO): Likewise. (STMT_VINFO_BB_VINFO): Likewise. * tree-vect-data-refs.c: Adjust for the above, adding vec_info * parameters and adjusting calls. * tree-vect-loop-manip.c: Likewise. * tree-vect-loop.c: Likewise. * tree-vect-patterns.c: Likewise. * tree-vect-slp.c: Likewise. * tree-vect-stmts.c: Likewise. * tree-vectorizer.c: Likewise. * target.def (add_stmt_cost): Add vec_info * parameter. * target.h (stmt_in_inner_loop_p): Likewise. * targhooks.c (default_add_stmt_cost): Adjust. * doc/tm.texi: Re-generate. * config/aarch64/aarch64.c (aarch64_extending_load_p): Add vec_info * parameter and adjust. (aarch64_sve_adjust_stmt_cost): Likewise. (aarch64_add_stmt_cost): Likewise. * config/arm/arm.c (arm_add_stmt_cost): Likewise. * config/i386/i386.c (ix86_add_stmt_cost): Likewise. * config/rs6000/rs6000.c (rs6000_add_stmt_cost): Likewise.
2020-01-20tree-optimization/93094 pass down VECTORIZED_CALL to versioningRichard Biener1-5/+6
When versioning is run the IL is already mangled and finding a VECTORIZED_CALL IFN can fail. 2020-01-20 Richard Biener <rguenther@suse.de> PR tree-optimization/93094 * tree-vectorizer.h (vect_loop_versioning): Adjust. (vect_transform_loop): Likewise. * tree-vectorizer.c (try_vectorize_loop_1): Pass down loop_vectorized_call to vect_transform_loop. * tree-vect-loop.c (vect_transform_loop): Pass down loop_vectorized_call to vect_loop_versioning. * tree-vect-loop-manip.c (vect_loop_versioning): Use the earlier discovered loop_vectorized_call. * gcc.dg/vect/pr93094.c: New testcase.
2020-01-10[vect] Keep track of DR_OFFSET advance in dr_vec_info rather than data_referenceAndre Vieira1-14/+9
gcc/ChangeLog: 2020-01-10 Andre Vieira <andre.simoesdiasvieira@arm.com> * tree-vect-data-refs.c (vect_create_addr_base_for_vector_ref): Use get_dr_vinfo_offset * tree-vect-loop.c (update_epilogue_loop_vinfo): Remove orig_drs_init parameter and its use to reset DR_OFFSET's. (vect_transform_loop): Remove orig_drs_init argument. * tree-vect-loop-manip.c (vect_update_init_of_dr): Update the offset member of dr_vec_info rather than the offset of the associated data_reference's innermost_loop_behavior. (vect_update_init_of_dr): Pass dr_vec_info instead of data_reference. (vect_do_peeling): Remove orig_drs_init parameter and its construction. * tree-vect-stmts.c (check_scan_store): Replace use of DR_OFFSET with get_dr_vinfo_offset. (vectorizable_store): Likewise. (vectorizable_load): Likewise. From-SVN: r280107
2020-01-01Update copyright years.Jakub Jelinek1-1/+1
From-SVN: r279813
2019-12-31Fix SSA update when vectorisation adds a vdef to a read-only loopRichard Sandiford1-4/+62
This patch fixes an awkward corner case in which: (a) we apply if-conversion to a loop; (b) the original scalar loop doesn't have a vdef, and thus doesn't need a virtual phi; (c) the vectorised main loop does need a vdef and a virtual phi (see below); (d) we also vectorise the epilogue; and (e) the vectorised epilogue still needs a scalar epilogue The specific case in which (c) applies is if a read-only loop is vectorised using IFN_LOAD_LANES, which uses clobber statements to mark the lifetime of the temporary array. The vectoriser relies on the SSA renamer to update virtual operands. All would probably be well if it postponed this update until after it had vectorised both the main loop and the epilogue loop. However, when vectorising the epilogue, vect_do_peeling does: create_lcssa_for_virtual_phi (loop); update_ssa (TODO_update_ssa_only_virtuals); (with "loop" in this case being the to-be-vectorised epilogue loop). So the vectoriser puts the virtual operand into SSA form for the vectorised main loop as a separate step, during the early stages of vectorising the epilogue. I wasn't sure at first why that update_ssa was there. It looked initially like it was related to create_lcssa_for_virtual_phi, which seemed strange when create_lcssa_for_virtual_phi keeps the SSA form up-to-date. But before r241099 it had the following comment, which AFAICT is still the reason: /* We might have a queued need to update virtual SSA form. As we delete the update SSA machinery below after doing a regular incremental SSA update during loop copying make sure we don't lose that fact. ??? Needing to update virtual SSA form by renaming is unfortunate but not all of the vectorizer code inserting new loads / stores properly assigns virtual operands to those statements. */ The patch restores that comment since IMO it's helpful. (a), (d) and (e) mean that we copy the original un-if-converted scalar loop to act as the scalar epilogue. The update_ssa above means that this copying needs to cope with any new virtual SSA names in the main loop. The code to do that (reasonably) assumed that one of two things was true: (1) the scalar loop and the vector loops don't have vdefs, and so no virtual operand update is needed. The definition that applies on entry to the loops is the same in all cases. (2) the scalar loop and the vector loops have virtual phis, and so -- after applying create_lcssa_for_virtual_phi on the to-be-vectorised epilogue loop -- the virtual operand update can be handled in the same way as for normal SSA names. But (b) and (c) together mean that the scalar loop and the still-to-be-vectorised epilogue loop have no virtual phi that (2) can use. We'd therefore keep the original vuses when duplicating, rather than updating them to the definition that applies on exit from the epilogue loop. (Since the epilogue is still unvectorised and has no vdefs, the definition that applies on exit is the same as the one that applies on entry.) This patch therefore adds a third case: the scalar loop and to-be-vectorised epilogue have no virtual defs, but the main loop does. 2019-12-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vect-loop-manip.c (create_lcssa_for_virtual_phi): Return the incoming virtual operand definition. (vect_do_peeling): When vectorizing an epilogue loop, handle the case in which the main loop has a virtual phi and the epilogue and scalar loops don't. Restore an earlier comment about the update_ssa call. gcc/testsuite/ * gcc.dg/vect/vect-epilogues-2.c: New test. From-SVN: r279802
2019-11-14Replace vec_info::vector_size with vec_info::vector_modeRichard Sandiford1-2/+3
This patch replaces vec_info::vector_size with vec_info::vector_mode, but for now continues to use it as a way of specifying a single vector size. This makes it easier for later patches to use related_vector_mode instead. 2019-11-14 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (vec_info::vector_size): Replace with... (vec_info::vector_mode): ...this new field. * tree-vect-loop.c (vect_update_vf_for_slp): Update accordingly. (vect_analyze_loop, vect_transform_loop): Likewise. * tree-vect-loop-manip.c (vect_do_peeling): Likewise. * tree-vect-slp.c (can_duplicate_and_interleave_p): Likewise. (vect_make_slp_decision, vect_slp_bb_region): Likewise. * tree-vect-stmts.c (get_vectype_for_scalar_type): Likewise. * tree-vectorizer.c (try_vectorize_loop_1): Likewise. gcc/testsuite/ * gcc.dg/vect/vect-tail-nomask-1.c: Update expected epilogue vectorization message. From-SVN: r278237
2019-11-14Add build_truth_vector_type_for_modeRichard Sandiford1-7/+13
Callers of vect_halve_mask_nunits and vect_double_mask_nunits already know what mode the resulting vector type should have, so we might as well create the vector type directly with that mode, just like build_vector_type_for_mode lets us build normal vectors with a known mode. This avoids the current awkwardness of having to recompute the mode starting from vec_info::vector_size, which hard-codes the assumption that all vectors have to be the same size. A later patch gets rid of build_truth_vector_type and build_same_sized_truth_vector_type, so the net effect of the series is to reduce the number of type functions by one. 2019-11-14 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree.h (build_truth_vector_type_for_mode): Declare. * tree.c (build_truth_vector_type_for_mode): New function, split out from... (build_truth_vector_type): ...here. (build_opaque_vector_type): Fix head comment. * tree-vectorizer.h (supportable_narrowing_operation): Remove vec_info parameter. (vect_halve_mask_nunits): Replace vec_info parameter with the mode of the new vector. (vect_double_mask_nunits): Likewise. * tree-vect-loop.c (vect_halve_mask_nunits): Likewise. (vect_double_mask_nunits): Likewise. * tree-vect-loop-manip.c: Include insn-config.h, rtl.h and recog.h. (vect_maybe_permute_loop_masks): Remove vinfo parameter. Update call to vect_halve_mask_nunits, getting the required mode from the unpack patterns. (vect_set_loop_condition_masked): Update call accordingly. * tree-vect-stmts.c (supportable_narrowing_operation): Remove vec_info parameter and update call to vect_double_mask_nunits. (vectorizable_conversion): Update call accordingly. (simple_integer_narrowing): Likewise. Remove vec_info parameter. (vectorizable_call): Update call accordingly. (supportable_widening_operation): Update call to vect_halve_mask_nunits. * config/aarch64/aarch64-sve-builtins.cc (register_builtin_types): Use build_truth_vector_type_mode instead of build_truth_vector_type. From-SVN: r278231
2019-11-13Avoid accounting for non-existent vector loop versioningRichard Sandiford1-2/+1
vect_analyze_loop_costing uses two profitability thresholds: a runtime one and a static compile-time one. The runtime one is simply the point at which the vector loop is cheaper than the scalar loop, while the static one also takes into account the cost of choosing between the scalar and vector loops at runtime. We compare this static cost against the expected execution frequency to decide whether it's worth generating any vector code at all. However, we never reclaimed the cost of applying the runtime threshold if it turned out that the vector code can always be used. And we only know whether that's true once we've calculated what the runtime threshold would be. 2019-11-13 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (vect_apply_runtime_profitability_check_p): New function. * tree-vect-loop-manip.c (vect_loop_versioning): Use it. * tree-vect-loop.c (vect_analyze_loop_2): Likewise. (vect_transform_loop): Likewise. (vect_analyze_loop_costing): Don't take the cost of versioning into account for the static profitability threshold if it turns out that no versioning is needed. From-SVN: r278124
2019-11-11[vect]Account for epilogue's peeling for gaps when checking if we have enoughAndre Vieira1-1/+4
niters for epilogue gcc/ChangeLog: 2019-11-11 Andre Vieira <andre.simoesdiasvieira@arm.com> * tree-vect-loop-manip.c (vect_do_peeling): Take epilogue gaps into account when checking if there are enough iterations to vectorize epilogue. gcc/testsuite/ChangeLog: 2019-11-11 Andre Vieira <andre.simoesdiasvieira@arm.com> * gcc.dg/vect/vect-reduc-epilogue-gaps.c: New test. From-SVN: r278049
2019-11-06[vect] PR92317: fix skip_epilogue creation for epiloguesAndre Vieira1-5/+7
gcc/ChangeLog: 2019-11-06 Andre Vieira <andre.simoesdiasvieira@arm.com> PR tree-optimization/92317 * tree-vect-loop-manip.c (slpeel_update_phi_nodes_for_guard2): Also update phi's with constant phi arguments. gcc/testsuite/ChangeLog: 2019-11-06 Andre Vieira <andre.simoesdiasvieira@arm.com> PR tree-optimization/92317 * gcc/testsuite/g++.dg/opt/pr92317.C: New test. From-SVN: r277877
2019-10-30re PR tree-optimization/92275 (ICE: error: definition in block 11 does not ↵Richard Biener1-23/+46
dominate use in block 15 since r277566) 2019-10-30 Richard Biener <rguenther@suse.de> PR tree-optimization/92275 * tree-vect-loop-manip.c (slpeel_update_phi_nodes_for_loops): Copy all loop-closed PHIs. * gcc.dg/torture/pr92275.c: New testcase. From-SVN: r277621
2019-10-29[vect]PR 88915: Vectorize epilogues when versioning loopsAndre Vieira1-27/+176
gcc/ChangeLog: 2019-10-29 Andre Vieira <andre.simoesdiasvieira@arm.com> PR 88915 * tree-ssa-loop-niter.h (simplify_replace_tree): Change declaration. * tree-ssa-loop-niter.c (simplify_replace_tree): Add context parameter and make the valueize function pointer also take a void pointer. * gcc/tree-ssa-sccvn.c (vn_valueize_wrapper): New function to wrap around vn_valueize, to call it without a context. (process_bb): Use vn_valueize_wrapper instead of vn_valueize. * tree-vect-loop.c (_loop_vec_info): Initialize epilogue_vinfos. (~_loop_vec_info): Release epilogue_vinfos. (vect_analyze_loop_costing): Use knowledge of main VF to estimate number of iterations of epilogue. (vect_analyze_loop_2): Adapt to analyse main loop for all supported vector sizes when vect-epilogues-nomask=1. Also keep track of lowest versioning threshold needed for main loop. (vect_analyze_loop): Likewise. (find_in_mapping): New helper function. (update_epilogue_loop_vinfo): New function. (vect_transform_loop): When vectorizing epilogues re-use analysis done on main loop and call update_epilogue_loop_vinfo to update it. * tree-vect-loop-manip.c (vect_update_inits_of_drs): No longer insert stmts on loop preheader edge. (vect_do_peeling): Enable skip-vectors when doing loop versioning if we decided to vectorize epilogues. Update epilogues NITERS and construct ADVANCE to update epilogues data references where needed. * tree-vectorizer.h (_loop_vec_info): Add epilogue_vinfos. (vect_do_peeling, vect_update_inits_of_drs, determine_peel_for_niter, vect_analyze_loop): Add or update declarations. * tree-vectorizer.c (try_vectorize_loop_1): Make sure to use already created loop_vec_info's for epilogues when available. Otherwise analyse epilogue separately. From-SVN: r277569
2019-10-21Pass a vec_info to vect_halve_mask_nunitsRichard Sandiford1-2/+2
2019-10-21 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (vect_halve_mask_nunits): Take a vec_info. * tree-vect-loop.c (vect_halve_mask_nunits): Likewise. * tree-vect-loop-manip.c (vect_maybe_permute_loop_masks): Update call accordingly. * tree-vect-stmts.c (supportable_widening_operation): Likewise. From-SVN: r277233
2019-10-21Pass a loop_vec_info to vect_maybe_permute_loop_masksRichard Sandiford1-2/+4
2019-10-21 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vect-loop-manip.c (vect_maybe_permute_loop_masks): Take a loop_vec_info. (vect_set_loop_condition_masked): Update call accordingly. From-SVN: r277232
2019-10-17[vect] Refactor versioning thresholdAndre Vieira1-4/+7
gcc/ChangeLog: 2019-10-17 Andre Vieira <andre.simoesdiasvieira@arm.com> * tree-vect-loop.c (vect_transform_loop): Move code from here... * tree-vect-loop-manip.c (vect_loop_versioning): ... to here. * tree-vectorizer.h (vect_loop_versioning): Remove unused parameters. From-SVN: r277101
2019-07-09PR c++/61339 - add mismatch between struct and class [-Wmismatched-tags] to ↵Martin Sebor1-36/+36
non-bugs gcc/c/ChangeLog: PR c++/61339 * c-decl.c (xref_tag): Change class-key of PODs to struct and others to class. (field_decl_cmp): Same. * c-parser.c (c_parser_struct_or_union_specifier): Same. * c-tree.h: Same. * gimple-parser.c (c_parser_gimple_compound_statement): Same. gcc/c-family/ChangeLog: PR c++/61339 * c-opts.c (handle_deferred_opts): : Change class-key of PODs to struct and others to class. * c-pretty-print.h: Same. gcc/cp/ChangeLog: PR c++/61339 * cp-tree.h: Change class-key of PODs to struct and others to class. * search.c: Same. * semantics.c (finalize_nrv_r): Same. gcc/lto/ChangeLog: PR c++/61339 * lto-common.c (lto_splay_tree_new): : Change class-key of PODs to struct and others to class. (mentions_vars_p): Same. (register_resolution): Same. (lto_register_var_decl_in_symtab): Same. (lto_register_function_decl_in_symtab): Same. (cmp_tree): Same. (lto_read_decls): Same. gcc/ChangeLog: PR c++/61339 * auto-profile.c: Change class-key of PODs to struct and others to class. * basic-block.h: Same. * bitmap.c (bitmap_alloc): Same. * bitmap.h: Same. * builtins.c (expand_builtin_prefetch): Same. (expand_builtin_interclass_mathfn): Same. (expand_builtin_strlen): Same. (expand_builtin_mempcpy_args): Same. (expand_cmpstr): Same. (expand_builtin___clear_cache): Same. (expand_ifn_atomic_bit_test_and): Same. (expand_builtin_thread_pointer): Same. (expand_builtin_set_thread_pointer): Same. * caller-save.c (setup_save_areas): Same. (replace_reg_with_saved_mem): Same. (insert_restore): Same. (insert_save): Same. (add_used_regs): Same. * cfg.c (get_bb_copy): Same. (set_loop_copy): Same. * cfg.h: Same. * cfganal.h: Same. * cfgexpand.c (alloc_stack_frame_space): Same. (add_stack_var): Same. (add_stack_var_conflict): Same. (add_scope_conflicts_1): Same. (update_alias_info_with_stack_vars): Same. (expand_used_vars): Same. * cfghooks.c (redirect_edge_and_branch_force): Same. (delete_basic_block): Same. (split_edge): Same. (make_forwarder_block): Same. (force_nonfallthru): Same. (duplicate_block): Same. (lv_flush_pending_stmts): Same. * cfghooks.h: Same. * cfgloop.c (flow_loops_cfg_dump): Same. (flow_loop_nested_p): Same. (superloop_at_depth): Same. (get_loop_latch_edges): Same. (flow_loop_dump): Same. (flow_loops_dump): Same. (flow_loops_free): Same. (flow_loop_nodes_find): Same. (establish_preds): Same. (flow_loop_tree_node_add): Same. (flow_loop_tree_node_remove): Same. (flow_loops_find): Same. (find_subloop_latch_edge_by_profile): Same. (find_subloop_latch_edge_by_ivs): Same. (mfb_redirect_edges_in_set): Same. (form_subloop): Same. (merge_latch_edges): Same. (disambiguate_multiple_latches): Same. (disambiguate_loops_with_multiple_latches): Same. (flow_bb_inside_loop_p): Same. (glb_enum_p): Same. (get_loop_body_with_size): Same. (get_loop_body): Same. (fill_sons_in_loop): Same. (get_loop_body_in_dom_order): Same. (get_loop_body_in_custom_order): Same. (release_recorded_exits): Same. (get_loop_exit_edges): Same. (num_loop_branches): Same. (remove_bb_from_loops): Same. (find_common_loop): Same. (delete_loop): Same. (cancel_loop): Same. (verify_loop_structure): Same. (loop_preheader_edge): Same. (loop_exit_edge_p): Same. (single_exit): Same. (loop_exits_to_bb_p): Same. (loop_exits_from_bb_p): Same. (get_loop_location): Same. (record_niter_bound): Same. (get_estimated_loop_iterations_int): Same. (max_stmt_executions_int): Same. (likely_max_stmt_executions_int): Same. (get_estimated_loop_iterations): Same. (get_max_loop_iterations): Same. (get_max_loop_iterations_int): Same. (get_likely_max_loop_iterations): Same. * cfgloop.h (simple_loop_desc): Same. (get_loop): Same. (loop_depth): Same. (loop_outer): Same. (loop_iterator::next): Same. (loop_outermost): Same. * cfgloopanal.c (mark_irreducible_loops): Same. (num_loop_insns): Same. (average_num_loop_insns): Same. (expected_loop_iterations_unbounded): Same. (expected_loop_iterations): Same. (mark_loop_exit_edges): Same. (single_likely_exit): Same. * cfgloopmanip.c (fix_bb_placement): Same. (fix_bb_placements): Same. (remove_path): Same. (place_new_loop): Same. (add_loop): Same. (scale_loop_frequencies): Same. (scale_loop_profile): Same. (create_empty_if_region_on_edge): Same. (create_empty_loop_on_edge): Same. (loopify): Same. (unloop): Same. (fix_loop_placements): Same. (copy_loop_info): Same. (duplicate_loop): Same. (duplicate_subloops): Same. (loop_redirect_edge): Same. (can_duplicate_loop_p): Same. (duplicate_loop_to_header_edge): Same. (mfb_keep_just): Same. (has_preds_from_loop): Same. (create_preheader): Same. (create_preheaders): Same. (lv_adjust_loop_entry_edge): Same. (loop_version): Same. * cfgloopmanip.h: Same. * cgraph.h: Same. * cgraphbuild.c: Same. * combine.c (make_extraction): Same. * config/i386/i386-features.c: Same. * config/i386/i386-features.h: Same. * config/i386/i386.c (ix86_emit_outlined_ms2sysv_save): Same. (ix86_emit_outlined_ms2sysv_restore): Same. (ix86_noce_conversion_profitable_p): Same. (ix86_init_cost): Same. (ix86_simd_clone_usable): Same. * configure.ac: Same. * coretypes.h: Same. * data-streamer-in.c (string_for_index): Same. (streamer_read_indexed_string): Same. (streamer_read_string): Same. (bp_unpack_indexed_string): Same. (bp_unpack_string): Same. (streamer_read_uhwi): Same. (streamer_read_hwi): Same. (streamer_read_gcov_count): Same. (streamer_read_wide_int): Same. * data-streamer.h (streamer_write_bitpack): Same. (bp_unpack_value): Same. (streamer_write_char_stream): Same. (streamer_write_hwi_in_range): Same. (streamer_write_record_start): Same. * ddg.c (create_ddg_dep_from_intra_loop_link): Same. (add_cross_iteration_register_deps): Same. (build_intra_loop_deps): Same. * df-core.c (df_analyze): Same. (loop_post_order_compute): Same. (loop_inverted_post_order_compute): Same. * df-problems.c (df_rd_alloc): Same. (df_rd_simulate_one_insn): Same. (df_rd_local_compute): Same. (df_rd_init_solution): Same. (df_rd_confluence_n): Same. (df_rd_transfer_function): Same. (df_rd_free): Same. (df_rd_dump_defs_set): Same. (df_rd_top_dump): Same. (df_lr_alloc): Same. (df_lr_reset): Same. (df_lr_local_compute): Same. (df_lr_init): Same. (df_lr_confluence_n): Same. (df_lr_free): Same. (df_lr_top_dump): Same. (df_lr_verify_transfer_functions): Same. (df_live_alloc): Same. (df_live_reset): Same. (df_live_init): Same. (df_live_confluence_n): Same. (df_live_finalize): Same. (df_live_free): Same. (df_live_top_dump): Same. (df_live_verify_transfer_functions): Same. (df_mir_alloc): Same. (df_mir_reset): Same. (df_mir_init): Same. (df_mir_confluence_n): Same. (df_mir_free): Same. (df_mir_top_dump): Same. (df_word_lr_alloc): Same. (df_word_lr_reset): Same. (df_word_lr_init): Same. (df_word_lr_confluence_n): Same. (df_word_lr_free): Same. (df_word_lr_top_dump): Same. (df_md_alloc): Same. (df_md_simulate_one_insn): Same. (df_md_reset): Same. (df_md_init): Same. (df_md_free): Same. (df_md_top_dump): Same. * df-scan.c (df_insn_delete): Same. (df_insn_rescan): Same. (df_notes_rescan): Same. (df_sort_and_compress_mws): Same. (df_install_mws): Same. (df_refs_add_to_chains): Same. (df_ref_create_structure): Same. (df_ref_record): Same. (df_def_record_1): Same. (df_find_hard_reg_defs): Same. (df_uses_record): Same. (df_get_conditional_uses): Same. (df_get_call_refs): Same. (df_recompute_luids): Same. (df_get_entry_block_def_set): Same. (df_entry_block_defs_collect): Same. (df_get_exit_block_use_set): Same. (df_exit_block_uses_collect): Same. (df_mws_verify): Same. (df_bb_verify): Same. * df.h (df_scan_get_bb_info): Same. * doc/tm.texi: Same. * dse.c (record_store): Same. * dumpfile.h: Same. * emit-rtl.c (const_fixed_hasher::equal): Same. (set_mem_attributes_minus_bitpos): Same. (change_address): Same. (adjust_address_1): Same. (offset_address): Same. * emit-rtl.h: Same. * except.c (dw2_build_landing_pads): Same. (sjlj_emit_dispatch_table): Same. * explow.c (allocate_dynamic_stack_space): Same. (emit_stack_probe): Same. (probe_stack_range): Same. * expmed.c (store_bit_field_using_insv): Same. (store_bit_field_1): Same. (store_integral_bit_field): Same. (extract_bit_field_using_extv): Same. (extract_bit_field_1): Same. (emit_cstore): Same. * expr.c (emit_block_move_via_cpymem): Same. (expand_cmpstrn_or_cmpmem): Same. (set_storage_via_setmem): Same. (emit_single_push_insn_1): Same. (expand_assignment): Same. (store_constructor): Same. (expand_expr_real_2): Same. (expand_expr_real_1): Same. (try_casesi): Same. * flags.h: Same. * function.c (try_fit_stack_local): Same. (assign_stack_local_1): Same. (assign_stack_local): Same. (cut_slot_from_list): Same. (insert_slot_to_list): Same. (max_slot_level): Same. (move_slot_to_level): Same. (temp_address_hasher::equal): Same. (remove_unused_temp_slot_addresses): Same. (assign_temp): Same. (combine_temp_slots): Same. (update_temp_slot_address): Same. (preserve_temp_slots): Same. * function.h: Same. * fwprop.c: Same. * gcc-rich-location.h: Same. * gcov.c: Same. * genattrtab.c (check_attr_test): Same. (check_attr_value): Same. (convert_set_attr_alternative): Same. (convert_set_attr): Same. (check_defs): Same. (copy_boolean): Same. (get_attr_value): Same. (expand_delays): Same. (make_length_attrs): Same. (min_fn): Same. (make_alternative_compare): Same. (simplify_test_exp): Same. (tests_attr_p): Same. (get_attr_order): Same. (clear_struct_flag): Same. (gen_attr): Same. (compares_alternatives_p): Same. (gen_insn): Same. (gen_delay): Same. (find_attrs_to_cache): Same. (write_test_expr): Same. (walk_attr_value): Same. (write_attr_get): Same. (eliminate_known_true): Same. (write_insn_cases): Same. (write_attr_case): Same. (write_attr_valueq): Same. (write_attr_value): Same. (write_dummy_eligible_delay): Same. (next_comma_elt): Same. (find_attr): Same. (make_internal_attr): Same. (copy_rtx_unchanging): Same. (gen_insn_reserv): Same. (check_tune_attr): Same. (make_automaton_attrs): Same. (handle_arg): Same. * genextract.c (gen_insn): Same. (VEC_char_to_string): Same. * genmatch.c (print_operand): Same. (lower): Same. (parser::parse_operation): Same. (parser::parse_capture): Same. (parser::parse_c_expr): Same. (parser::parse_simplify): Same. (main): Same. * genoutput.c (output_operand_data): Same. (output_get_insn_name): Same. (compare_operands): Same. (place_operands): Same. (process_template): Same. (validate_insn_alternatives): Same. (validate_insn_operands): Same. (gen_expand): Same. (note_constraint): Same. * genpreds.c (write_one_predicate_function): Same. (add_constraint): Same. (process_define_register_constraint): Same. (write_lookup_constraint_1): Same. (write_lookup_constraint_array): Same. (write_insn_constraint_len): Same. (write_reg_class_for_constraint_1): Same. (write_constraint_satisfied_p_array): Same. * genrecog.c (optimize_subroutine_group): Same. * gensupport.c (process_define_predicate): Same. (queue_pattern): Same. (remove_from_queue): Same. (process_rtx): Same. (is_predicable): Same. (change_subst_attribute): Same. (subst_pattern_match): Same. (alter_constraints): Same. (alter_attrs_for_insn): Same. (shift_output_template): Same. (alter_output_for_subst_insn): Same. (process_one_cond_exec): Same. (subst_dup): Same. (process_define_cond_exec): Same. (mnemonic_htab_callback): Same. (gen_mnemonic_attr): Same. (read_md_rtx): Same. * ggc-page.c: Same. * gimple-loop-interchange.cc (dump_reduction): Same. (dump_induction): Same. (loop_cand::~loop_cand): Same. (free_data_refs_with_aux): Same. (tree_loop_interchange::interchange_loops): Same. (tree_loop_interchange::map_inductions_to_loop): Same. (tree_loop_interchange::move_code_to_inner_loop): Same. (compute_access_stride): Same. (compute_access_strides): Same. (proper_loop_form_for_interchange): Same. (tree_loop_interchange_compute_ddrs): Same. (prune_datarefs_not_in_loop): Same. (prepare_data_references): Same. (pass_linterchange::execute): Same. * gimple-loop-jam.c (bb_prevents_fusion_p): Same. (unroll_jam_possible_p): Same. (fuse_loops): Same. (adjust_unroll_factor): Same. (tree_loop_unroll_and_jam): Same. * gimple-loop-versioning.cc (loop_versioning::~loop_versioning): Same. (loop_versioning::expensive_stmt_p): Same. (loop_versioning::version_for_unity): Same. (loop_versioning::dump_inner_likelihood): Same. (loop_versioning::find_per_loop_multiplication): Same. (loop_versioning::analyze_term_using_scevs): Same. (loop_versioning::record_address_fragment): Same. (loop_versioning::analyze_expr): Same. (loop_versioning::analyze_blocks): Same. (loop_versioning::prune_conditions): Same. (loop_versioning::merge_loop_info): Same. (loop_versioning::add_loop_to_queue): Same. (loop_versioning::decide_whether_loop_is_versionable): Same. (loop_versioning::make_versioning_decisions): Same. (loop_versioning::implement_versioning_decisions): Same. * gimple-ssa-evrp-analyze.c (evrp_range_analyzer::record_ranges_from_phis): Same. * gimple-ssa-store-merging.c (split_store::split_store): Same. (count_multiple_uses): Same. (split_group): Same. (imm_store_chain_info::output_merged_store): Same. (pass_store_merging::process_store): Same. * gimple-ssa-strength-reduction.c (slsr_process_phi): Same. * gimple-ssa-warn-alloca.c (adjusted_warn_limit): Same. (is_max): Same. (alloca_call_type): Same. (pass_walloca::execute): Same. * gimple-streamer-in.c (input_phi): Same. (input_gimple_stmt): Same. * gimple-streamer.h: Same. * godump.c (go_force_record_alignment): Same. (go_format_type): Same. (go_output_type): Same. (go_output_fndecl): Same. (go_output_typedef): Same. (keyword_hash_init): Same. (find_dummy_types): Same. * graph.c (draw_cfg_nodes_no_loops): Same. (draw_cfg_nodes_for_loop): Same. * hard-reg-set.h (hard_reg_set_iter_next): Same. * hsa-brig.c: Same. * hsa-common.h (hsa_internal_fn_hasher::equal): Same. * hsa-dump.c (dump_hsa_cfun): Same. * hsa-gen.c (gen_function_def_parameters): Same. * hsa-regalloc.c (dump_hsa_cfun_regalloc): Same. * input.c (dump_line_table_statistics): Same. (test_lexer): Same. * input.h: Same. * internal-fn.c (get_multi_vector_move): Same. (expand_load_lanes_optab_fn): Same. (expand_GOMP_SIMT_ENTER_ALLOC): Same. (expand_GOMP_SIMT_EXIT): Same. (expand_GOMP_SIMT_LAST_LANE): Same. (expand_GOMP_SIMT_ORDERED_PRED): Same. (expand_GOMP_SIMT_VOTE_ANY): Same. (expand_GOMP_SIMT_XCHG_BFLY): Same. (expand_GOMP_SIMT_XCHG_IDX): Same. (expand_addsub_overflow): Same. (expand_neg_overflow): Same. (expand_mul_overflow): Same. (expand_call_mem_ref): Same. (expand_mask_load_optab_fn): Same. (expand_scatter_store_optab_fn): Same. (expand_gather_load_optab_fn): Same. * ipa-cp.c (ipa_get_parm_lattices): Same. (print_all_lattices): Same. (ignore_edge_p): Same. (build_toporder_info): Same. (free_toporder_info): Same. (push_node_to_stack): Same. (ipcp_lattice<valtype>::set_contains_variable): Same. (set_agg_lats_to_bottom): Same. (ipcp_bits_lattice::meet_with): Same. (set_single_call_flag): Same. (initialize_node_lattices): Same. (ipa_get_jf_ancestor_result): Same. (ipcp_verify_propagated_values): Same. (propagate_scalar_across_jump_function): Same. (propagate_context_across_jump_function): Same. (propagate_bits_across_jump_function): Same. (ipa_vr_operation_and_type_effects): Same. (propagate_vr_across_jump_function): Same. (set_check_aggs_by_ref): Same. (set_chain_of_aglats_contains_variable): Same. (merge_aggregate_lattices): Same. (agg_pass_through_permissible_p): Same. (propagate_aggs_across_jump_function): Same. (call_passes_through_thunk_p): Same. (propagate_constants_across_call): Same. (devirtualization_time_bonus): Same. (good_cloning_opportunity_p): Same. (context_independent_aggregate_values): Same. (gather_context_independent_values): Same. (perform_estimation_of_a_value): Same. (estimate_local_effects): Same. (value_topo_info<valtype>::add_val): Same. (add_all_node_vals_to_toposort): Same. (value_topo_info<valtype>::propagate_effects): Same. (ipcp_propagate_stage): Same. (ipcp_discover_new_direct_edges): Same. (same_node_or_its_all_contexts_clone_p): Same. (cgraph_edge_brings_value_p): Same. (gather_edges_for_value): Same. (create_specialized_node): Same. (find_more_scalar_values_for_callers_subset): Same. (find_more_contexts_for_caller_subset): Same. (copy_plats_to_inter): Same. (intersect_aggregates_with_edge): Same. (find_aggregate_values_for_callers_subset): Same. (cgraph_edge_brings_all_agg_vals_for_node): Same. (decide_about_value): Same. (decide_whether_version_node): Same. (spread_undeadness): Same. (identify_dead_nodes): Same. (ipcp_store_vr_results): Same. * ipa-devirt.c (final_warning_record::grow_type_warnings): Same. * ipa-fnsummary.c (ipa_fn_summary::account_size_time): Same. (redirect_to_unreachable): Same. (edge_set_predicate): Same. (evaluate_conditions_for_known_args): Same. (evaluate_properties_for_edge): Same. (ipa_fn_summary_t::duplicate): Same. (ipa_call_summary_t::duplicate): Same. (dump_ipa_call_summary): Same. (ipa_dump_fn_summary): Same. (eliminated_by_inlining_prob): Same. (set_cond_stmt_execution_predicate): Same. (set_switch_stmt_execution_predicate): Same. (compute_bb_predicates): Same. (will_be_nonconstant_expr_predicate): Same. (phi_result_unknown_predicate): Same. (analyze_function_body): Same. (compute_fn_summary): Same. (estimate_edge_devirt_benefit): Same. (estimate_edge_size_and_time): Same. (estimate_calls_size_and_time): Same. (estimate_node_size_and_time): Same. (remap_edge_change_prob): Same. (remap_edge_summaries): Same. (ipa_merge_fn_summary_after_inlining): Same. (ipa_fn_summary_generate): Same. (inline_read_section): Same. (ipa_fn_summary_read): Same. (ipa_fn_summary_write): Same. * ipa-fnsummary.h: Same. * ipa-hsa.c (ipa_hsa_read_section): Same. * ipa-icf-gimple.c (func_checker::compare_loops): Same. * ipa-icf.c (sem_function::param_used_p): Same. * ipa-inline-analysis.c (do_estimate_edge_time): Same. * ipa-inline.c (edge_badness): Same. (inline_small_functions): Same. * ipa-polymorphic-call.c (ipa_polymorphic_call_context::stream_out): Same. * ipa-predicate.c (predicate::remap_after_duplication): Same. (predicate::remap_after_inlining): Same. (predicate::stream_out): Same. * ipa-predicate.h: Same. * ipa-profile.c (ipa_profile_read_summary): Same. * ipa-prop.c (ipa_get_param_decl_index_1): Same. (count_formal_params): Same. (ipa_dump_param): Same. (ipa_alloc_node_params): Same. (ipa_print_node_jump_functions_for_edge): Same. (ipa_print_node_jump_functions): Same. (ipa_load_from_parm_agg): Same. (get_ancestor_addr_info): Same. (ipa_compute_jump_functions_for_edge): Same. (ipa_analyze_virtual_call_uses): Same. (ipa_analyze_stmt_uses): Same. (ipa_analyze_params_uses_in_bb): Same. (update_jump_functions_after_inlining): Same. (try_decrement_rdesc_refcount): Same. (ipa_impossible_devirt_target): Same. (update_indirect_edges_after_inlining): Same. (combine_controlled_uses_counters): Same. (ipa_edge_args_sum_t::duplicate): Same. (ipa_write_jump_function): Same. (ipa_write_indirect_edge_info): Same. (ipa_write_node_info): Same. (ipa_read_edge_info): Same. (ipa_prop_read_section): Same. (read_replacements_section): Same. * ipa-prop.h (ipa_get_param_count): Same. (ipa_get_param): Same. (ipa_get_type): Same. (ipa_get_param_move_cost): Same. (ipa_set_param_used): Same. (ipa_get_controlled_uses): Same. (ipa_set_controlled_uses): Same. (ipa_get_cs_argument_count): Same. * ipa-pure-const.c (analyze_function): Same. (pure_const_read_summary): Same. * ipa-ref.h: Same. * ipa-reference.c (ipa_reference_read_optimization_summary): Same. * ipa-split.c (test_nonssa_use): Same. (dump_split_point): Same. (dominated_by_forbidden): Same. (split_part_set_ssa_name_p): Same. (find_split_points): Same. * ira-build.c (finish_loop_tree_nodes): Same. (low_pressure_loop_node_p): Same. * ira-color.c (ira_reuse_stack_slot): Same. * ira-int.h: Same. * ira.c (setup_reg_equiv): Same. (print_insn_chain): Same. (ira): Same. * loop-doloop.c (doloop_condition_get): Same. (add_test): Same. (record_reg_sets): Same. (doloop_optimize): Same. * loop-init.c (loop_optimizer_init): Same. (fix_loop_structure): Same. * loop-invariant.c (merge_identical_invariants): Same. (compute_always_reached): Same. (find_exits): Same. (may_assign_reg_p): Same. (find_invariants_bb): Same. (find_invariants_body): Same. (replace_uses): Same. (can_move_invariant_reg): Same. (free_inv_motion_data): Same. (move_single_loop_invariants): Same. (change_pressure): Same. (mark_ref_regs): Same. (calculate_loop_reg_pressure): Same. * loop-iv.c (biv_entry_hasher::equal): Same. (iv_extend_to_rtx_code): Same. (check_iv_ref_table_size): Same. (clear_iv_info): Same. (latch_dominating_def): Same. (iv_get_reaching_def): Same. (iv_constant): Same. (iv_subreg): Same. (iv_extend): Same. (iv_neg): Same. (iv_add): Same. (iv_mult): Same. (get_biv_step): Same. (record_iv): Same. (analyzed_for_bivness_p): Same. (record_biv): Same. (iv_analyze_biv): Same. (iv_analyze_expr): Same. (iv_analyze_def): Same. (iv_analyze_op): Same. (iv_analyze): Same. (iv_analyze_result): Same. (biv_p): Same. (eliminate_implied_conditions): Same. (simplify_using_initial_values): Same. (shorten_into_mode): Same. (canonicalize_iv_subregs): Same. (determine_max_iter): Same. (check_simple_exit): Same. (find_simple_exit): Same. (get_simple_loop_desc): Same. * loop-unroll.c (report_unroll): Same. (decide_unrolling): Same. (unroll_loops): Same. (loop_exit_at_end_p): Same. (decide_unroll_constant_iterations): Same. (unroll_loop_constant_iterations): Same. (compare_and_jump_seq): Same. (unroll_loop_runtime_iterations): Same. (decide_unroll_stupid): Same. (unroll_loop_stupid): Same. (referenced_in_one_insn_in_loop_p): Same. (reset_debug_uses_in_loop): Same. (analyze_iv_to_split_insn): Same. * lra-eliminations.c (lra_debug_elim_table): Same. (setup_can_eliminate): Same. (form_sum): Same. (lra_get_elimination_hard_regno): Same. (lra_eliminate_regs_1): Same. (eliminate_regs_in_insn): Same. (update_reg_eliminate): Same. (init_elimination): Same. (lra_eliminate): Same. * lra-int.h: Same. * lra-lives.c (initiate_live_solver): Same. * lra-remat.c (create_remat_bb_data): Same. * lra-spills.c (lra_spill): Same. * lra.c (lra_set_insn_recog_data): Same. (lra_set_used_insn_alternative_by_uid): Same. (init_reg_info): Same. (expand_reg_info): Same. * lto-cgraph.c (output_symtab): Same. (read_identifier): Same. (get_alias_symbol): Same. (input_node): Same. (input_varpool_node): Same. (input_ref): Same. (input_edge): Same. (input_cgraph_1): Same. (input_refs): Same. (input_symtab): Same. (input_offload_tables): Same. (output_cgraph_opt_summary): Same. (input_edge_opt_summary): Same. (input_cgraph_opt_section): Same. * lto-section-in.c (lto_free_raw_section_data): Same. (lto_create_simple_input_block): Same. (lto_free_function_in_decl_state_for_node): Same. * lto-streamer-in.c (lto_tag_check_set): Same. (lto_location_cache::revert_location_cache): Same. (lto_location_cache::input_location): Same. (lto_input_location): Same. (stream_input_location_now): Same. (lto_input_tree_ref): Same. (lto_input_eh_catch_list): Same. (input_eh_region): Same. (lto_init_eh): Same. (make_new_block): Same. (input_cfg): Same. (fixup_call_stmt_edges): Same. (input_struct_function_base): Same. (input_function): Same. (lto_read_body_or_constructor): Same. (lto_read_tree_1): Same. (lto_read_tree): Same. (lto_input_scc): Same. (lto_input_tree_1): Same. (lto_input_toplevel_asms): Same. (lto_input_mode_table): Same. (lto_reader_init): Same. (lto_data_in_create): Same. * lto-streamer-out.c (output_cfg): Same. * lto-streamer.h: Same. * modulo-sched.c (duplicate_insns_of_cycles): Same. (generate_prolog_epilog): Same. (mark_loop_unsched): Same. (dump_insn_location): Same. (loop_canon_p): Same. (sms_schedule): Same. * omp-expand.c (expand_omp_for_ordered_loops): Same. (expand_omp_for_generic): Same. (expand_omp_for_static_nochunk): Same. (expand_omp_for_static_chunk): Same. (expand_omp_simd): Same. (expand_omp_taskloop_for_inner): Same. (expand_oacc_for): Same. (expand_omp_atomic_pipeline): Same. (mark_loops_in_oacc_kernels_region): Same. * omp-offload.c (oacc_xform_loop): Same. * omp-simd-clone.c (simd_clone_adjust): Same. * optabs-query.c (get_traditional_extraction_insn): Same. * optabs.c (expand_vector_broadcast): Same. (expand_binop_directly): Same. (expand_twoval_unop): Same. (expand_twoval_binop): Same. (expand_unop_direct): Same. (emit_indirect_jump): Same. (emit_conditional_move): Same. (emit_conditional_neg_or_complement): Same. (emit_conditional_add): Same. (vector_compare_rtx): Same. (expand_vec_perm_1): Same. (expand_vec_perm_const): Same. (expand_vec_cond_expr): Same. (expand_vec_series_expr): Same. (maybe_emit_atomic_exchange): Same. (maybe_emit_sync_lock_test_and_set): Same. (expand_atomic_compare_and_swap): Same. (expand_atomic_load): Same. (expand_atomic_store): Same. (maybe_emit_op): Same. (valid_multiword_target_p): Same. (create_integer_operand): Same. (maybe_legitimize_operand_same_code): Same. (maybe_legitimize_operand): Same. (create_convert_operand_from_type): Same. (can_reuse_operands_p): Same. (maybe_legitimize_operands): Same. (maybe_gen_insn): Same. (maybe_expand_insn): Same. (maybe_expand_jump_insn): Same. (expand_insn): Same. * optabs.h (create_expand_operand): Same. (create_fixed_operand): Same. (create_output_operand): Same. (create_input_operand): Same. (create_convert_operand_to): Same. (create_convert_operand_from): Same. * optinfo.h: Same. * poly-int.h: Same. * predict.c (optimize_insn_for_speed_p): Same. (optimize_loop_for_size_p): Same. (optimize_loop_for_speed_p): Same. (optimize_loop_nest_for_speed_p): Same. (get_base_value): Same. (predicted_by_loop_heuristics_p): Same. (predict_extra_loop_exits): Same. (predict_loops): Same. (predict_paths_for_bb): Same. (predict_paths_leading_to): Same. (propagate_freq): Same. (pass_profile::execute): Same. * predict.h: Same. * profile-count.c (profile_count::differs_from_p): Same. (profile_probability::differs_lot_from_p): Same. * profile-count.h: Same. * profile.c (branch_prob): Same. * regrename.c (free_chain_data): Same. (mark_conflict): Same. (create_new_chain): Same. (merge_overlapping_regs): Same. (init_rename_info): Same. (merge_chains): Same. (regrename_analyze): Same. (regrename_do_replace): Same. (scan_rtx_reg): Same. (record_out_operands): Same. (build_def_use): Same. * regrename.h: Same. * reload.h: Same. * reload1.c (init_reload): Same. (maybe_fix_stack_asms): Same. (copy_reloads): Same. (count_pseudo): Same. (count_spilled_pseudo): Same. (find_reg): Same. (find_reload_regs): Same. (select_reload_regs): Same. (spill_hard_reg): Same. (fixup_eh_region_note): Same. (set_reload_reg): Same. (allocate_reload_reg): Same. (compute_reload_subreg_offset): Same. (reload_adjust_reg_for_icode): Same. (emit_input_reload_insns): Same. (emit_output_reload_insns): Same. (do_input_reload): Same. (inherit_piecemeal_p): Same. * rtl.h: Same. * sanopt.c (maybe_get_dominating_check): Same. (maybe_optimize_ubsan_ptr_ifn): Same. (can_remove_asan_check): Same. (maybe_optimize_asan_check_ifn): Same. (sanopt_optimize_walker): Same. * sched-deps.c (add_dependence_list): Same. (chain_to_prev_insn): Same. (add_insn_mem_dependence): Same. (create_insn_reg_set): Same. (maybe_extend_reg_info_p): Same. (sched_analyze_reg): Same. (sched_analyze_1): Same. (get_implicit_reg_pending_clobbers): Same. (chain_to_prev_insn_p): Same. (deps_analyze_insn): Same. (deps_start_bb): Same. (sched_free_deps): Same. (init_deps): Same. (init_deps_reg_last): Same. (free_deps): Same. * sched-ebb.c: Same. * sched-int.h: Same. * sched-rgn.c (add_branch_dependences): Same. (concat_insn_mem_list): Same. (deps_join): Same. (sched_rgn_compute_dependencies): Same. * sel-sched-ir.c (reset_target_context): Same. (copy_deps_context): Same. (init_id_from_df): Same. (has_dependence_p): Same. (change_loops_latches): Same. (bb_top_order_comparator): Same. (make_region_from_loop_preheader): Same. (sel_init_pipelining): Same. (get_loop_nest_for_rgn): Same. (make_regions_from_the_rest): Same. (sel_is_loop_preheader_p): Same. * sel-sched-ir.h (inner_loop_header_p): Same. (get_all_loop_exits): Same. * selftest.h: Same. * sese.c (sese_build_liveouts): Same. (sese_insert_phis_for_liveouts): Same. * sese.h (defined_in_sese_p): Same. * sreal.c (sreal::stream_out): Same. * sreal.h: Same. * streamer-hooks.h: Same. * target-globals.c (save_target_globals): Same. * target-globals.h: Same. * target.def: Same. * target.h: Same. * targhooks.c (default_has_ifunc_p): Same. (default_empty_mask_is_expensive): Same. (default_init_cost): Same. * targhooks.h: Same. * toplev.c: Same. * tree-affine.c (aff_combination_mult): Same. (aff_combination_expand): Same. (aff_combination_constant_multiple_p): Same. * tree-affine.h: Same. * tree-cfg.c (build_gimple_cfg): Same. (replace_loop_annotate_in_block): Same. (replace_uses_by): Same. (remove_bb): Same. (dump_cfg_stats): Same. (gimple_duplicate_sese_region): Same. (gimple_duplicate_sese_tail): Same. (move_block_to_fn): Same. (replace_block_vars_by_duplicates): Same. (move_sese_region_to_fn): Same. (print_loops_bb): Same. (print_loop): Same. (print_loops): Same. (debug): Same. (debug_loops): Same. * tree-cfg.h: Same. * tree-chrec.c (chrec_fold_plus_poly_poly): Same. (chrec_fold_multiply_poly_poly): Same. (chrec_evaluate): Same. (chrec_component_in_loop_num): Same. (reset_evolution_in_loop): Same. (is_multivariate_chrec): Same. (chrec_contains_symbols): Same. (nb_vars_in_chrec): Same. (chrec_convert_1): Same. (chrec_convert_aggressive): Same. * tree-chrec.h: Same. * tree-core.h: Same. * tree-data-ref.c (dump_data_dependence_relation): Same. (canonicalize_base_object_address): Same. (data_ref_compare_tree): Same. (prune_runtime_alias_test_list): Same. (get_segment_min_max): Same. (create_intersect_range_checks): Same. (conflict_fn_no_dependence): Same. (object_address_invariant_in_loop_p): Same. (analyze_ziv_subscript): Same. (analyze_siv_subscript_cst_affine): Same. (analyze_miv_subscript): Same. (analyze_overlapping_iterations): Same. (build_classic_dist_vector_1): Same. (add_other_self_distances): Same. (same_access_functions): Same. (build_classic_dir_vector): Same. (subscript_dependence_tester_1): Same. (subscript_dependence_tester): Same. (access_functions_are_affine_or_constant_p): Same. (get_references_in_stmt): Same. (loop_nest_has_data_refs): Same. (graphite_find_data_references_in_stmt): Same. (find_data_references_in_bb): Same. (get_base_for_alignment): Same. (find_loop_nest_1): Same. (find_loop_nest): Same. * tree-data-ref.h (dr_alignment): Same. (ddr_dependence_level): Same. * tree-if-conv.c (fold_build_cond_expr): Same. (add_to_predicate_list): Same. (add_to_dst_predicate_list): Same. (phi_convertible_by_degenerating_args): Same. (idx_within_array_bound): Same. (all_preds_critical_p): Same. (pred_blocks_visited_p): Same. (predicate_bbs): Same. (build_region): Same. (if_convertible_loop_p_1): Same. (is_cond_scalar_reduction): Same. (predicate_scalar_phi): Same. (remove_conditions_and_labels): Same. (combine_blocks): Same. (version_loop_for_if_conversion): Same. (versionable_outer_loop_p): Same. (ifcvt_local_dce): Same. (tree_if_conversion): Same. (pass_if_conversion::gate): Same. * tree-if-conv.h: Same. * tree-inline.c (maybe_move_debug_stmts_to_successors): Same. * tree-loop-distribution.c (bb_top_order_cmp): Same. (free_rdg): Same. (stmt_has_scalar_dependences_outside_loop): Same. (copy_loop_before): Same. (create_bb_after_loop): Same. (const_with_all_bytes_same): Same. (generate_memset_builtin): Same. (generate_memcpy_builtin): Same. (destroy_loop): Same. (build_rdg_partition_for_vertex): Same. (compute_access_range): Same. (data_ref_segment_size): Same. (latch_dominated_by_data_ref): Same. (compute_alias_check_pairs): Same. (fuse_memset_builtins): Same. (finalize_partitions): Same. (find_seed_stmts_for_distribution): Same. (prepare_perfect_loop_nest): Same. * tree-parloops.c (lambda_transform_legal_p): Same. (loop_parallel_p): Same. (reduc_stmt_res): Same. (add_field_for_name): Same. (create_call_for_reduction_1): Same. (replace_uses_in_bb_by): Same. (transform_to_exit_first_loop_alt): Same. (try_transform_to_exit_first_loop_alt): Same. (transform_to_exit_first_loop): Same. (num_phis): Same. (gen_parallel_loop): Same. (gather_scalar_reductions): Same. (get_omp_data_i_param): Same. (try_create_reduction_list): Same. (oacc_entry_exit_single_gang): Same. (parallelize_loops): Same. * tree-pass.h: Same. * tree-predcom.c (determine_offset): Same. (last_always_executed_block): Same. (split_data_refs_to_components): Same. (suitable_component_p): Same. (valid_initializer_p): Same. (find_looparound_phi): Same. (insert_looparound_copy): Same. (add_looparound_copies): Same. (determine_roots_comp): Same. (predcom_tmp_var): Same. (initialize_root_vars): Same. (initialize_root_vars_store_elim_1): Same. (initialize_root_vars_store_elim_2): Same. (finalize_eliminated_stores): Same. (initialize_root_vars_lm): Same. (remove_stmt): Same. (determine_unroll_factor): Same. (execute_pred_commoning_cbck): Same. (base_names_in_chain_on): Same. (combine_chains): Same. (pcom_stmt_dominates_stmt_p): Same. (try_combine_chains): Same. (prepare_initializers_chain_store_elim): Same. (prepare_initializers_chain): Same. (prepare_initializers): Same. (prepare_finalizers_chain): Same. (prepare_finalizers): Same. (insert_init_seqs): Same. * tree-scalar-evolution.c (loop_phi_node_p): Same. (compute_overall_effect_of_inner_loop): Same. (add_to_evolution_1): Same. (add_to_evolution): Same. (follow_ssa_edge_binary): Same. (follow_ssa_edge_expr): Same. (backedge_phi_arg_p): Same. (follow_ssa_edge_in_condition_phi_branch): Same. (follow_ssa_edge_in_condition_phi): Same. (follow_ssa_edge_inner_loop_phi): Same. (follow_ssa_edge): Same. (analyze_evolution_in_loop): Same. (analyze_initial_condition): Same. (interpret_loop_phi): Same. (interpret_condition_phi): Same. (interpret_rhs_expr): Same. (interpret_expr): Same. (interpret_gimple_assign): Same. (analyze_scalar_evolution_1): Same. (analyze_scalar_evolution): Same. (analyze_scalar_evolution_for_address_of): Same. (get_instantiated_value_entry): Same. (loop_closed_phi_def): Same. (instantiate_scev_name): Same. (instantiate_scev_poly): Same. (instantiate_scev_binary): Same. (instantiate_scev_convert): Same. (instantiate_scev_not): Same. (instantiate_scev_r): Same. (instantiate_scev): Same. (resolve_mixers): Same. (initialize_scalar_evolutions_analyzer): Same. (scev_reset_htab): Same. (scev_reset): Same. (derive_simple_iv_with_niters): Same. (simple_iv_with_niters): Same. (expression_expensive_p): Same. (final_value_replacement_loop): Same. * tree-scalar-evolution.h (block_before_loop): Same. * tree-ssa-address.h: Same. * tree-ssa-dce.c (find_obviously_necessary_stmts): Same. * tree-ssa-dom.c (edge_info::record_simple_equiv): Same. (record_edge_info): Same. * tree-ssa-live.c (var_map_base_fini): Same. (remove_unused_locals): Same. * tree-ssa-live.h: Same. * tree-ssa-loop-ch.c (should_duplicate_loop_header_p): Same. (pass_ch_vect::execute): Same. (pass_ch::process_loop_p): Same. * tree-ssa-loop-im.c (mem_ref_hasher::hash): Same. (movement_possibility): Same. (outermost_invariant_loop): Same. (stmt_cost): Same. (determine_max_movement): Same. (invariantness_dom_walker::before_dom_children): Same. (move_computations): Same. (may_move_till): Same. (force_move_till_op): Same. (force_move_till): Same. (memref_free): Same. (record_mem_ref_loc): Same. (set_ref_stored_in_loop): Same. (mark_ref_stored): Same. (sort_bbs_in_loop_postorder_cmp): Same. (sort_locs_in_loop_postorder_cmp): Same. (analyze_memory_references): Same. (mem_refs_may_alias_p): Same. (find_ref_loc_in_loop_cmp): Same. (rewrite_mem_ref_loc::operator): Same. (first_mem_ref_loc_1::operator): Same. (sm_set_flag_if_changed::operator): Same. (execute_sm_if_changed_flag_set): Same. (execute_sm): Same. (hoist_memory_references): Same. (ref_always_accessed::operator): Same. (refs_independent_p): Same. (record_dep_loop): Same. (ref_indep_loop_p_1): Same. (ref_indep_loop_p): Same. (can_sm_ref_p): Same. (find_refs_for_sm): Same. (loop_suitable_for_sm): Same. (store_motion_loop): Same. (store_motion): Same. (fill_always_executed_in): Same. * tree-ssa-loop-ivcanon.c (constant_after_peeling): Same. (estimated_unrolled_size): Same. (loop_edge_to_cancel): Same. (remove_exits_and_undefined_stmts): Same. (remove_redundant_iv_tests): Same. (unloop_loops): Same. (estimated_peeled_sequence_size): Same. (try_peel_loop): Same. (canonicalize_loop_induction_variables): Same. (canonicalize_induction_variables): Same. * tree-ssa-loop-ivopts.c (iv_inv_expr_hasher::equal): Same. (name_info): Same. (stmt_after_inc_pos): Same. (contains_abnormal_ssa_name_p): Same. (niter_for_exit): Same. (find_bivs): Same. (mark_bivs): Same. (find_givs_in_bb): Same. (find_induction_variables): Same. (find_interesting_uses_cond): Same. (outermost_invariant_loop_for_expr): Same. (idx_find_step): Same. (add_candidate_1): Same. (add_iv_candidate_derived_from_uses): Same. (alloc_use_cost_map): Same. (prepare_decl_rtl): Same. (generic_predict_doloop_p): Same. (computation_cost): Same. (determine_common_wider_type): Same. (get_computation_aff_1): Same. (get_use_type): Same. (determine_group_iv_cost_address): Same. (iv_period): Same. (difference_cannot_overflow_p): Same. (may_eliminate_iv): Same. (determine_set_costs): Same. (cheaper_cost_pair): Same. (compare_cost_pair): Same. (iv_ca_cand_for_group): Same. (iv_ca_recount_cost): Same. (iv_ca_set_remove_invs): Same. (iv_ca_set_no_cp): Same. (iv_ca_set_add_invs): Same. (iv_ca_set_cp): Same. (iv_ca_add_group): Same. (iv_ca_cost): Same. (iv_ca_compare_deps): Same. (iv_ca_delta_reverse): Same. (iv_ca_delta_commit): Same. (iv_ca_cand_used_p): Same. (iv_ca_delta_free): Same. (iv_ca_new): Same. (iv_ca_free): Same. (iv_ca_dump): Same. (iv_ca_extend): Same. (iv_ca_narrow): Same. (iv_ca_prune): Same. (cheaper_cost_with_cand): Same. (iv_ca_replace): Same. (try_add_cand_for): Same. (get_initial_solution): Same. (try_improve_iv_set): Same. (find_optimal_iv_set_1): Same. (create_new_iv): Same. (rewrite_use_compare): Same. (remove_unused_ivs): Same. (determine_scaling_factor): Same. * tree-ssa-loop-ivopts.h: Same. * tree-ssa-loop-manip.c (create_iv): Same. (compute_live_loop_exits): Same. (add_exit_phi): Same. (add_exit_phis): Same. (find_uses_to_rename_use): Same. (find_uses_to_rename_def): Same. (find_uses_to_rename_in_loop): Same. (rewrite_into_loop_closed_ssa): Same. (check_loop_closed_ssa_bb): Same. (split_loop_exit_edge): Same. (ip_end_pos): Same. (ip_normal_pos): Same. (copy_phi_node_args): Same. (gimple_duplicate_loop_to_header_edge): Same. (can_unroll_loop_p): Same. (determine_exit_conditions): Same. (scale_dominated_blocks_in_loop): Same. (niter_for_unrolled_loop): Same. (tree_transform_and_unroll_loop): Same. (rewrite_all_phi_nodes_with_iv): Same. * tree-ssa-loop-manip.h: Same. * tree-ssa-loop-niter.c (number_of_iterations_ne_max): Same. (number_of_iterations_ne): Same. (assert_no_overflow_lt): Same. (assert_loop_rolls_lt): Same. (number_of_iterations_lt): Same. (adjust_cond_for_loop_until_wrap): Same. (tree_simplify_using_condition): Same. (simplify_using_initial_conditions): Same. (simplify_using_outer_evolutions): Same. (loop_only_exit_p): Same. (ssa_defined_by_minus_one_stmt_p): Same. (number_of_iterations_popcount): Same. (number_of_iterations_exit): Same. (find_loop_niter): Same. (finite_loop_p): Same. (chain_of_csts_start): Same. (get_val_for): Same. (loop_niter_by_eval): Same. (derive_constant_upper_bound_ops): Same. (do_warn_aggressive_loop_optimizations): Same. (record_estimate): Same. (get_cst_init_from_scev): Same. (record_nonwrapping_iv): Same. (idx_infer_loop_bounds): Same. (infer_loop_bounds_from_ref): Same. (infer_loop_bounds_from_array): Same. (infer_loop_bounds_from_pointer_arith): Same. (infer_loop_bounds_from_signedness): Same. (bound_index): Same. (discover_iteration_bound_by_body_walk): Same. (maybe_lower_iteration_bound): Same. (estimate_numbers_of_iterations): Same. (estimated_loop_iterations): Same. (estimated_loop_iterations_int): Same. (max_loop_iterations): Same. (max_loop_iterations_int): Same. (likely_max_loop_iterations): Same. (likely_max_loop_iterations_int): Same. (estimated_stmt_executions_int): Same. (max_stmt_executions): Same. (likely_max_stmt_executions): Same. (estimated_stmt_executions): Same. (stmt_dominates_stmt_p): Same. (nowrap_type_p): Same. (loop_exits_before_overflow): Same. (scev_var_range_cant_overflow): Same. (scev_probably_wraps_p): Same. (free_numbers_of_iterations_estimates): Same. * tree-ssa-loop-niter.h: Same. * tree-ssa-loop-prefetch.c (release_mem_refs): Same. (idx_analyze_ref): Same. (analyze_ref): Same. (gather_memory_references_ref): Same. (mark_nontemporal_store): Same. (emit_mfence_after_loop): Same. (may_use_storent_in_loop_p): Same. (mark_nontemporal_stores): Same. (should_unroll_loop_p): Same. (volume_of_dist_vector): Same. (add_subscript_strides): Same. (self_reuse_distance): Same. (insn_to_prefetch_ratio_too_small_p): Same. * tree-ssa-loop-split.c (split_at_bb_p): Same. (patch_loop_exit): Same. (find_or_create_guard_phi): Same. (easy_exit_values): Same. (connect_loop_phis): Same. (connect_loops): Same. (compute_new_first_bound): Same. (split_loop): Same. (tree_ssa_split_loops): Same. * tree-ssa-loop-unswitch.c (tree_ssa_unswitch_loops): Same. (is_maybe_undefined): Same. (tree_may_unswitch_on): Same. (simplify_using_entry_checks): Same. (tree_unswitch_single_loop): Same. (tree_unswitch_loop): Same. (tree_unswitch_outer_loop): Same. (empty_bb_without_guard_p): Same. (used_outside_loop_p): Same. (get_vop_from_header): Same. (hoist_guard): Same. * tree-ssa-loop.c (gate_oacc_kernels): Same. (get_lsm_tmp_name): Same. * tree-ssa-loop.h: Same. * tree-ssa-reassoc.c (add_repeat_to_ops_vec): Same. (build_and_add_sum): Same. (no_side_effect_bb): Same. (get_ops): Same. (linearize_expr): Same. (should_break_up_subtract): Same. (linearize_expr_tree): Same. * tree-ssa-scopedtables.c: Same. * tree-ssa-scopedtables.h: Same. * tree-ssa-structalias.c (condense_visit): Same. (label_visit): Same. (dump_pred_graph): Same. (perform_var_substitution): Same. (move_complex_constraints): Same. (remove_preds_and_fake_succs): Same. * tree-ssa-threadupdate.c (dbds_continue_enumeration_p): Same. (determine_bb_domination_status): Same. (duplicate_thread_path): Same. (thread_through_all_blocks): Same. * tree-ssa-threadupdate.h: Same. * tree-streamer-in.c (streamer_read_string_cst): Same. (input_identifier): Same. (unpack_ts_type_common_value_fields): Same. (unpack_ts_block_value_fields): Same. (unpack_ts_translation_unit_decl_value_fields): Same. (unpack_ts_omp_clause_value_fields): Same. (streamer_read_tree_bitfields): Same. (streamer_alloc_tree): Same. (lto_input_ts_common_tree_pointers): Same. (lto_input_ts_vector_tree_pointers): Same. (lto_input_ts_poly_tree_pointers): Same. (lto_input_ts_complex_tree_pointers): Same. (lto_input_ts_decl_minimal_tree_pointers): Same. (lto_input_ts_decl_common_tree_pointers): Same. (lto_input_ts_decl_non_common_tree_pointers): Same. (lto_input_ts_decl_with_vis_tree_pointers): Same. (lto_input_ts_field_decl_tree_pointers): Same. (lto_input_ts_function_decl_tree_pointers): Same. (lto_input_ts_type_common_tree_pointers): Same. (lto_input_ts_type_non_common_tree_pointers): Same. (lto_input_ts_list_tree_pointers): Same. (lto_input_ts_vec_tree_pointers): Same. (lto_input_ts_exp_tree_pointers): Same. (lto_input_ts_block_tree_pointers): Same. (lto_input_ts_binfo_tree_pointers): Same. (lto_input_ts_constructor_tree_pointers): Same. (lto_input_ts_omp_clause_tree_pointers): Same. (streamer_read_tree_body): Same. * tree-streamer.h: Same. * tree-switch-conversion.c (bit_test_cluster::is_beneficial): Same. * tree-vect-data-refs.c (vect_get_smallest_scalar_type): Same. (vect_analyze_possibly_independent_ddr): Same. (vect_analyze_data_ref_dependence): Same. (vect_compute_data_ref_alignment): Same. (vect_enhance_data_refs_alignment): Same. (vect_analyze_data_ref_access): Same. (vect_check_gather_scatter): Same. (vect_find_stmt_data_reference): Same. (vect_create_addr_base_for_vector_ref): Same. (vect_setup_realignment): Same. (vect_supportable_dr_alignment): Same. * tree-vect-loop-manip.c (rename_variables_in_bb): Same. (adjust_phi_and_debug_stmts): Same. (vect_set_loop_mask): Same. (add_preheader_seq): Same. (vect_maybe_permute_loop_masks): Same. (vect_set_loop_masks_directly): Same. (vect_set_loop_condition_masked): Same. (vect_set_loop_condition_unmasked): Same. (slpeel_duplicate_current_defs_from_edges): Same. (slpeel_add_loop_guard): Same. (slpeel_can_duplicate_loop_p): Same. (create_lcssa_for_virtual_phi): Same. (iv_phi_p): Same. (vect_update_ivs_after_vectorizer): Same. (vect_gen_vector_loop_niters_mult_vf): Same. (slpeel_update_phi_nodes_for_loops): Same. (slpeel_update_phi_nodes_for_guard1): Same. (find_guard_arg): Same. (slpeel_update_phi_nodes_for_guard2): Same. (slpeel_update_phi_nodes_for_lcssa): Same. (vect_do_peeling): Same. (vect_create_cond_for_alias_checks): Same. (vect_loop_versioning): Same. * tree-vect-loop.c (vect_determine_vf_for_stmt): Same. (vect_inner_phi_in_double_reduction_p): Same. (vect_analyze_scalar_cycles_1): Same. (vect_fixup_scalar_cycles_with_patterns): Same. (vect_get_loop_niters): Same. (bb_in_loop_p): Same. (vect_get_max_nscalars_per_iter): Same. (vect_verify_full_masking): Same. (vect_compute_single_scalar_iteration_cost): Same. (vect_analyze_loop_form_1): Same. (vect_analyze_loop_form): Same. (vect_active_double_reduction_p): Same. (vect_analyze_loop_operations): Same. (neutral_op_for_slp_reduction): Same. (vect_is_simple_reduction): Same. (vect_model_reduction_cost): Same. (get_initial_def_for_reduction): Same. (get_initial_defs_for_reduction): Same. (vect_create_epilog_for_reduction): Same. (vectorize_fold_left_reduction): Same. (vectorizable_reduction): Same. (vectorizable_induction): Same. (vectorizable_live_operation): Same. (loop_niters_no_overflow): Same. (vect_get_loop_mask): Same. (vect_transform_loop_stmt): Same. (vect_transform_loop): Same. * tree-vect-patterns.c (vect_reassociating_reduction_p): Same. (vect_determine_precisions): Same. (vect_pattern_recog_1): Same. * tree-vect-slp.c (vect_analyze_slp_instance): Same. * tree-vect-stmts.c (stmt_vectype): Same. (process_use): Same. (vect_init_vector_1): Same. (vect_truncate_gather_scatter_offset): Same. (get_group_load_store_type): Same. (vect_build_gather_load_calls): Same. (vect_get_strided_load_store_ops): Same. (vectorizable_simd_clone_call): Same. (vectorizable_store): Same. (permute_vec_elements): Same. (vectorizable_load): Same. (vect_transform_stmt): Same. (supportable_widening_operation): Same. * tree-vectorizer.c (vec_info::replace_stmt): Same. (vec_info::free_stmt_vec_info): Same. (vect_free_loop_info_assumptions): Same. (vect_loop_vectorized_call): Same. (set_uid_loop_bbs): Same. (vectorize_loops): Same. * tree-vectorizer.h (STMT_VINFO_BB_VINFO): Same. * tree.c (add_tree_to_fld_list): Same. (fld_type_variant_equal_p): Same. (fld_decl_context): Same. (fld_incomplete_type_of): Same. (free_lang_data_in_binfo): Same. (need_assembler_name_p): Same. (find_decls_types_r): Same. (get_eh_types_for_runtime): Same. (find_decls_types_in_eh_region): Same. (find_decls_types_in_node): Same. (assign_assembler_name_if_needed): Same. * value-prof.c (stream_out_histogram_value): Same. * value-prof.h: Same. * var-tracking.c (use_narrower_mode): Same. (prepare_call_arguments): Same. (vt_expand_loc_callback): Same. (resolve_expansions_pending_recursion): Same. (vt_expand_loc): Same. * varasm.c (const_hash_1): Same. (compare_constant): Same. (tree_output_constant_def): Same. (simplify_subtraction): Same. (get_pool_constant): Same. (output_constant_pool_2): Same. (output_constant_pool_1): Same. (mark_constants_in_pattern): Same. (mark_constant_pool): Same. (get_section_anchor): Same. * vr-values.c (compare_range_with_value): Same. (vr_values::extract_range_from_phi_node): Same. * vr-values.h: Same. * web.c (unionfind_union): Same. * wide-int.h: Same. From-SVN: r273311
2019-07-04re PR tree-optimization/90911 (456.hmmer regression with r272239)Richard Biener1-2/+11
2019-07-04 Richard Biener <rguenther@suse.de> PR tree-optimization/90911 * tree-vectorizer.h (_loop_vec_info::scalar_loop_scaling): New field. (LOOP_VINFO_SCALAR_LOOP_SCALING): new. * tree-vect-loop.c (_loop_vec_info::_loop_vec_info): Initialize scalar_loop_scaling. (vect_transform_loop): Scale scalar loop profile if needed. * tree-vect-loop-manip.c (vect_loop_versioning): When re-using the loop copy from if-conversion adjust edge probabilities and scale the vectorized loop body profile, queue the scalar profile for updating after peeling. From-SVN: r273082
2019-06-21re PR tree-optimization/90913 (ICE in maybe_gen_insn, at optabs.c:7341 since ↵Richard Biener1-2/+5
r272239) 2019-06-21 Richard Biener <rguenther@suse.de> PR tree-optimization/90913 * tree-vect-loop-manip.c (vect_loop_versioning): Do not re-use the scalar variant of if-conversion versioning. * gfortran.dg/vect/pr90913.f90: New testcase. From-SVN: r272545
2019-06-18Restore correct iv step for fully-masked loopsRichard Sandiford1-27/+22
r272233 introduced a large number of execution failures on SVE. The patch hard-coded an IV step of VF, but for SLP groups it needs to be VF * group size. Also, iv_precision had type widest_int but only needs to be unsigned int. 2019-06-18 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vect-loop-manip.c (vect_set_loop_masks_directly): Remove vf parameter. Restore the previous iv step of nscalars_step, but give it iv_type rather than compare_type. Tweak code order to match the comments. (vect_set_loop_condition_masked): Update accordingly. * tree-vect-loop.c (vect_verify_full_masking): Use "unsigned int" for iv_precision. Tweak comment formatting. From-SVN: r272411
2019-06-13tree-vectorizer.h (vect_loop_vectorized_call): Declare.Richard Biener1-78/+128
2019-06-13 Richard Biener <rguenther@suse.de> * tree-vectorizer.h (vect_loop_vectorized_call): Declare. * tree-vectorizer.c (vect_loop_vectorized_call): Export and also return the condition stmt. * tree-vect-loop-manip.c (vect_loop_versioning): Compute outermost loop we can version and version that, reusing the loop version created by if-conversion instead of versioning again. * gcc.dg/vect/vect-version-1.c: New testcase. * gcc.dg/vect/vect-version-2.c: Likewise. From-SVN: r272239
2019-06-13re PR target/88838 ([SVE] Use 32-bit WHILELO in LP64 mode)Kugan Vivekanandarajah1-31/+15
gcc/ChangeLog: 2019-06-13 Kugan Vivekanandarajah <kugan.vivekanandarajah@linaro.org> PR target/88838 * tree-vect-loop-manip.c (vect_set_loop_masks_directly): If the compare_type is not with Pmode size, we will create an IV with Pmode size with truncated use (i.e. converted to the correct type). * tree-vect-loop.c (vect_verify_full_masking): Find IV type. (vect_iv_limit_for_full_masking): New. Factored out of vect_set_loop_condition_masked. * tree-vectorizer.h (LOOP_VINFO_MASK_IV_TYPE): New. (vect_iv_limit_for_full_masking): Declare. gcc/testsuite/ChangeLog: 2019-06-13 Kugan Vivekanandarajah <kugan.vivekanandarajah@linaro.org> PR target/88838 * gcc.target/aarch64/pr88838.c: New test. * gcc.target/aarch64/sve/while_1.c: Adjust. From-SVN: r272233
2019-05-16omp-low.c (lower_rec_input_clauses): If OMP_CLAUSE_IF has non-constant ↵Jakub Jelinek1-0/+27
expression... * omp-low.c (lower_rec_input_clauses): If OMP_CLAUSE_IF has non-constant expression, force sctx.lane and use two argument IFN_GOMP_SIMD_LANE instead of single argument. * tree-ssa-dce.c (eliminate_unnecessary_stmts): Don't DCE two argument IFN_GOMP_SIMD_LANE without lhs. * tree-vectorizer.h (struct _loop_vec_info): Add simd_if_cond member. (LOOP_VINFO_SIMD_IF_COND, LOOP_REQUIRES_VERSIONING_FOR_SIMD_IF_COND): Define. (LOOP_REQUIRES_VERSIONING): Or in LOOP_REQUIRES_VERSIONING_FOR_SIMD_IF_COND. * tree-vect-loop.c (_loop_vec_info::_loop_vec_info): Initialize simd_if_cond. (vect_analyze_loop_2): Punt if LOOP_VINFO_SIMD_IF_COND is constant 0. * tree-vect-loop-manip.c (vect_loop_versioning): Add runtime check from simd if clause if needed. * gcc.dg/vect/vect-simd-1.c: New test. * gcc.dg/vect/vect-simd-2.c: New test. * gcc.dg/vect/vect-simd-3.c: New test. * gcc.dg/vect/vect-simd-4.c: New test. From-SVN: r271298
2019-03-11re PR tree-optimization/89649 (r269458 FAILs g++.dg/pr80481.C, ↵Richard Biener1-1/+5
scan-assembler-not vmovaps) 2019-03-11 Richard Biener <rguenther@suse.de> PR tree-optimization/89649 * tree-vectorizer.h (vect_loop_versioning): Adjust prototype. * tree-vect-loop-manip.c (vect_do_peeling): Unset force_vectorize on the prolog and epilog loops. (vect_loop_versioning): Return copy of loop. * tree-vect-loop.c (vect_transform_loop): Unset force_vectorize on the non-vectorized version of the loop. From-SVN: r269578
2019-01-01Update copyright years.Jakub Jelinek1-1/+1
From-SVN: r267494
2018-11-13Eliminate source_location in favor of location_tDavid Malcolm1-1/+1
Historically GCC used location_t, while libcpp used source_location. This inconsistency has been annoying me for a while, so this patch removes source_location in favor of location_t throughout (as the latter is shorter). gcc/ChangeLog: * builtins.c: Replace "source_location" with "location_t". * diagnostic-show-locus.c: Likewise. * diagnostic.c: Likewise. * dumpfile.c: Likewise. * gcc-rich-location.h: Likewise. * genmatch.c: Likewise. * gimple.h: Likewise. * gimplify.c: Likewise. * input.c: Likewise. * input.h: Likewise. Eliminate the typedef. * omp-expand.c: Likewise. * selftest.h: Likewise. * substring-locations.h (get_source_location_for_substring): Rename to.. (get_location_within_string): ...this. * tree-cfg.c: Replace "source_location" with "location_t". * tree-cfgcleanup.c: Likewise. * tree-diagnostic.c: Likewise. * tree-into-ssa.c: Likewise. * tree-outof-ssa.c: Likewise. * tree-parloops.c: Likewise. * tree-phinodes.c: Likewise. * tree-phinodes.h: Likewise. * tree-ssa-loop-ivopts.c: Likewise. * tree-ssa-loop-manip.c: Likewise. * tree-ssa-phiopt.c: Likewise. * tree-ssa-phiprop.c: Likewise. * tree-ssa-threadupdate.c: Likewise. * tree-ssa.c: Likewise. * tree-ssa.h: Likewise. * tree-vect-loop-manip.c: Likewise. gcc/c-family/ChangeLog: * c-common.c (c_get_substring_location): Update for renaming of get_source_location_for_substring to get_location_within_string. * c-lex.c: Replace "source_location" with "location_t". * c-opts.c: Likewise. * c-ppoutput.c: Likewise. gcc/c/ChangeLog: * c-decl.c: Replace "source_location" with "location_t". * c-tree.h: Likewise. * c-typeck.c: Likewise. * gimple-parser.c: Likewise. gcc/cp/ChangeLog: * call.c: Replace "source_location" with "location_t". * cp-tree.h: Likewise. * cvt.c: Likewise. * name-lookup.c: Likewise. * parser.c: Likewise. * typeck.c: Likewise. gcc/fortran/ChangeLog: * cpp.c: Replace "source_location" with "location_t". * gfortran.h: Likewise. gcc/go/ChangeLog: * go-gcc-diagnostics.cc: Replace "source_location" with "location_t". * go-gcc.cc: Likewise. * go-linemap.cc: Likewise. * go-location.h: Likewise. * gofrontend/README: Likewise. gcc/jit/ChangeLog: * jit-playback.c: Replace "source_location" with "location_t". gcc/testsuite/ChangeLog: * g++.dg/plugin/comment_plugin.c: Replace "source_location" with "location_t". * gcc.dg/plugin/diagnostic_plugin_test_show_locus.c: Likewise. libcc1/ChangeLog: * libcc1plugin.cc: Replace "source_location" with "location_t". (plugin_context::get_source_location): Rename to... (plugin_context::get_location_t): ...this. * libcp1plugin.cc: Likewise. libcpp/ChangeLog: * charset.c: Replace "source_location" with "location_t". * directives-only.c: Likewise. * directives.c: Likewise. * errors.c: Likewise. * expr.c: Likewise. * files.c: Likewise. * include/cpplib.h: Likewise. Rename MAX_SOURCE_LOCATION to MAX_LOCATION_T. * include/line-map.h: Likewise. * init.c: Likewise. * internal.h: Likewise. * lex.c: Likewise. * line-map.c: Likewise. * location-example.txt: Likewise. * macro.c: Likewise. * pch.c: Likewise. * traditional.c: Likewise. From-SVN: r266085
2018-11-13[PATCH][GCC] Make DR_TARGET_ALIGNMENT compile time variableAndre Vieira1-7/+34
This patch enables targets to describe DR_TARGET_ALIGNMENT as a compile-time variable. It does so by turning the variable into a 'poly_uint64'. gcc/ChangeLog: 2018-11-13 Andre Vieira <andre.simoesdiasvieira@arm.com> * config/aarch64/aarch64.c (aarch64_vectorize_preferred_vector_alignment): Change return type to poly_uint64. (aarch64_simd_vector_alignment_reachable): Adapt to preferred vector alignment being a poly int. * doc/tm.texi (TARGET_VECTORIZE_PREFERRED_VECTOR_ALIGNMENT): Change return type to poly_uint64. * target.def (default_preferred_vector_alignment): Likewise. * targhooks.c (default_preferred_vector_alignment): Likewise. * targhooks.h (default_preferred_vector_alignment): Likewise. * tree-vect-data-refs.c (vect_calculate_target_alignment): Likewise. (vect_compute_data_ref_alignment): Adapt to vector alignment being a poly int. (vect_update_misalignment_for_peel): Likewise. (vect_enhance_data_refs_alignment): Likewise. (vect_find_same_alignment_drs): Likewise. (vect_duplicate_ssa_name_ptr_info): Likewise. (vect_setup_realignment): Likewise. (vect_can_force_dr_alignment_p): Change alignment parameter type to poly_uint64. * tree-vect-loop-manip.c (get_misalign_in_elems): Learn to construct a mask with a compile time variable vector alignment. (vect_gen_prolog_loop_niters): Adapt to vector alignment being a poly int. (vect_do_peeling): Exit early if vector alignment is not constant. * tree-vect-stmts.c (ensure_base_align): Adapt to vector alignment being a poly int. (vectorizable_store): Likewise. (vectorizable_load): Likweise. * tree-vectorizer.h (struct dr_vec_info): Make target_alignment field a poly_uint64. (vect_known_alignment_in_bytes): Adapt to vector alignment being a poly int. (vect_can_force_dr_alignment_p): Change alignment parameter type to poly_uint64. From-SVN: r266072
2018-11-06re PR tree-optimization/87889 (CPU2000 177.mesa failed to build)Richard Biener1-1/+2
2018-11-06 Richard Biener <rguenther@suse.de> PR tree-optimization/87889 * tree-vect-loop-manip.c (slpeel_duplicate_current_defs_from_edges): Do nothing if old and new arg are the same * gcc.dg/pr87894.c: New testcase. From-SVN: r265833
2018-11-05re PR tree-optimization/87873 (ICE: verify_gimple failed (error: ↵Richard Biener1-2/+7
incompatible types in PHI argument 0)) 2018-11-05 Richard Biener <rguenther@suse.de> PR tree-optimization/87873 * tree-ssa-loop-manip.h (split_loop_exit_edge): Add copy_constants_p argument. * tree-ssa-loop-manip.c (split_loop_exit_edge): Likewise. * tree-vect-loop.c (vect_transform_loop): When splitting the loop exit also create forwarder PHIs for constants. * tree-vect-loop-manip.c (slpeel_duplicate_current_defs_from_edges): Handle constant to_arg, add extra checking we match up the correct PHIs. * gcc.dg/pr87873.c: New testcase. From-SVN: r265812
2018-10-04Add -fopt-info-internalsDavid Malcolm1-2/+4
This patch introduces a verbosity level to dump messages: "user-facing" vs "internals". By default, messages at the top-level dump scope are "user-facing", whereas those that are in nested scopes are implicitly "internals", and are filtered out by -fopt-info unless a new "-internals" sub-option of "-fopt-info" is supplied (intended purely for use by GCC developers). Dumpfiles are unaffected by the change. Given that the vectorizer is the only subsystem using AUTO_DUMP_SCOPE (via DUMP_VECT_SCOPE), this only affects the vectorizer. Filtering out these implementation-detail messages goes a long way towards making -fopt-info-vec-all more accessible to advanced end-users; the follow-up patch restores the most pertinent missing details. gcc/ChangeLog: * doc/invoke.texi (-fopt-info): Document new "internals" sub-option. * dump-context.h (dump_context::apply_dump_filter_p): New decl. * dumpfile.c (dump_options): Update for renaming of MSG_ALL to MSG_ALL_KINDS. (optinfo_verbosity_options): Add "internals". (kind_as_string): Update for renaming of MSG_ALL to MSG_ALL_KINDS. (dump_context::apply_dump_filter_p): New member function. (dump_context::dump_loc): Use apply_dump_filter_p rather than explicitly masking the dump_kind. (dump_context::begin_scope): Increment the scope depth first. Use apply_dump_filter_p rather than explicitly masking the dump_kind. (dump_context::emit_item): Use apply_dump_filter_p rather than explicitly masking the dump_kind. (dump_dec): Likewise. (dump_hex): Likewise. (dump_switch_p_1): Default to MSG_ALL_PRIORITIES. (opt_info_switch_p_1): Default to MSG_PRIORITY_USER_FACING. (opt_info_switch_p): Update handling of default MSG_OPTIMIZED_LOCATIONS to cope with default of MSG_PRIORITY_USER_FACING. (dump_basic_block): Use apply_dump_filter_p rather than explicitly masking the dump_kind. (selftest::test_capture_of_dump_calls): Update test_dump_context instances to use MSG_ALL_KINDS | MSG_PRIORITY_USER_FACING rather than MSG_ALL. Generalize scope test to be run at all four combinations of with/without MSG_PRIORITY_USER_FACING and MSG_PRIORITY_INTERNALS, adding examples of explicit priority for each of the two values. * dumpfile.h (enum dump_flag): Add comment about the MSG_* flags. Rename MSG_ALL to MSG_ALL_KINDS. Add MSG_PRIORITY_USER_FACING, MSG_PRIORITY_INTERNALS, and MSG_ALL_PRIORITIES, updating the values for TDF_COMPARE_DEBUG and TDF_ALL_VALUES. (AUTO_DUMP_SCOPE): Add a note to the comment about the interaction with MSG_PRIORITY_*. * tree-vect-loop-manip.c (vect_loop_versioning): Mark versioning dump messages as MSG_PRIORITY_USER_FACING. * tree-vectorizer.h (DUMP_VECT_SCOPE): Add a note to the comment about the interaction with MSG_PRIORITY_*. gcc/testsuite/ChangeLog: * gcc.dg/plugin/dump-1.c: Update expected output for test_scopes due to "-internals" not being selected. * gcc.dg/plugin/dump-2.c: New test, based on dump-1.c, with "-internals" added to re-enable the output from test_scopes. * gcc.dg/plugin/plugin.exp (plugin_test_list): Add dump-2.c. From-SVN: r264851
2018-09-19dump_printf: use %T and %G throughoutDavid Malcolm1-25/+11
As promised at Cauldron, this patch uses %T and %G with dump_printf and dump_printf_loc calls to eliminate calls to dump_generic_expr (MSG_*, arg, TDF_SLIM) (via %T) and dump_gimple_stmt (MSG_*, TDF_SLIM, stmt, 0) (via %G) throughout the middle-end, simplifying numerous dump callsites. A few calls to these functions didn't match the above pattern; I didn't touch these. I wasn't able to use %E anywhere. gcc/ChangeLog: * tree-data-ref.c (runtime_alias_check_p): Use formatted printing with %T in place of calls to dump_generic_expr. (prune_runtime_alias_test_list): Likewise. (create_runtime_alias_checks): Likewise. * tree-vect-data-refs.c (vect_check_nonzero_value): Likewise. (vect_analyze_data_ref_dependence): Likewise. (vect_slp_analyze_data_ref_dependence): Likewise. (vect_record_base_alignment): Likewise. Use %G in place of call to dump_gimple_stmt. (vect_compute_data_ref_alignment): Likewise. (verify_data_ref_alignment): Likewise. (vect_find_same_alignment_drs): Likewise. (vect_analyze_group_access_1): Likewise. (vect_analyze_data_ref_accesses): Likewise. (dependence_distance_ge_vf): Likewise. (dump_lower_bound): Likewise. (vect_prune_runtime_alias_test_list): Likewise. (vect_find_stmt_data_reference): Likewise. (vect_analyze_data_refs): Likewise. (vect_create_addr_base_for_vector_ref): Likewise. (vect_create_data_ref_ptr): Likewise. * tree-vect-loop-manip.c (vect_set_loop_condition): Likewise. (vect_can_advance_ivs_p): Likewise. (vect_update_ivs_after_vectorizer): Likewise. (vect_gen_prolog_loop_niters): Likewise. (vect_prepare_for_masked_peels): Likewise. * tree-vect-loop.c (vect_determine_vf_for_stmt): Likewise. (vect_determine_vectorization_factor): Likewise. (vect_is_simple_iv_evolution): Likewise. (vect_analyze_scalar_cycles_1): Likewise. (vect_analyze_loop_operations): Likewise. (report_vect_op): Likewise. (vect_is_slp_reduction): Likewise. (check_reduction_path): Likewise. (vect_is_simple_reduction): Likewise. (vect_create_epilog_for_reduction): Likewise. (vect_finalize_reduction:): Likewise. (vectorizable_induction): Likewise. (vect_transform_loop_stmt): Likewise. (vect_transform_loop): Likewise. (optimize_mask_stores): Likewise. * tree-vect-patterns.c (vect_pattern_detected): Likewise. (vect_split_statement): Likewise. (vect_recog_over_widening_pattern): Likewise. (vect_recog_average_pattern): Likewise. (vect_determine_min_output_precision_1): Likewise. (vect_determine_precisions_from_range): Likewise. (vect_determine_precisions_from_users): Likewise. (vect_mark_pattern_stmts): Likewise. (vect_pattern_recog_1): Likewise. * tree-vect-slp.c (vect_get_and_check_slp_defs): Likewise. (vect_record_max_nunits): Likewise. (vect_build_slp_tree_1): Likewise. (vect_build_slp_tree_2): Likewise. (vect_print_slp_tree): Likewise. (vect_analyze_slp_instance): Likewise. (vect_detect_hybrid_slp_stmts): Likewise. (vect_detect_hybrid_slp_1): Likewise. (vect_slp_analyze_operations): Likewise. (vect_slp_analyze_bb_1): Likewise. (vect_transform_slp_perm_load): Likewise. (vect_schedule_slp_instance): Likewise. * tree-vect-stmts.c (vect_mark_relevant): Likewise. (vect_mark_stmts_to_be_vectorized): Likewise. (vect_init_vector_1): Likewise. (vect_get_vec_def_for_operand): Likewise. (vect_finish_stmt_generation_1): Likewise. (vect_check_load_store_mask): Likewise. (vectorizable_call): Likewise. (vectorizable_conversion): Likewise. (vectorizable_operation): Likewise. (vectorizable_load): Likewise. (vect_analyze_stmt): Likewise. (vect_is_simple_use): Likewise. (vect_get_vector_types_for_stmt): Likewise. (vect_get_mask_type_for_stmt): Likewise. * tree-vectorizer.c (increase_alignment): Likewise. From-SVN: r264424
2018-07-31[41/46] Add vec_info::remove_stmtRichard Sandiford1-2/+6
This patch adds a new helper function for permanently removing a statement and its associated stmt_vec_info. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (vec_info::remove_stmt): Declare. * tree-vectorizer.c (vec_info::remove_stmt): New function. * tree-vect-loop-manip.c (vect_set_loop_condition): Use it. * tree-vect-loop.c (vect_transform_loop): Likewise. * tree-vect-slp.c (vect_schedule_slp): Likewise. * tree-vect-stmts.c (vect_remove_stores): Likewise. From-SVN: r263156
2018-07-31[40/46] Add vec_info::lookup_drRichard Sandiford1-2/+2
This patch replaces DR_VECT_AUX and vect_dr_stmt with a new vec_info::lookup_dr function, so that the lookup is relative to a particular vec_info rather than to global state. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (vec_info::lookup_dr): New member function. (vect_dr_stmt): Delete. * tree-vectorizer.c (vec_info::lookup_dr): New function. * tree-vect-loop-manip.c (vect_update_inits_of_drs): Use it instead of DR_VECT_AUX. * tree-vect-data-refs.c (vect_analyze_possibly_independent_ddr) (vect_analyze_data_ref_dependence, vect_record_base_alignments) (vect_verify_datarefs_alignment, vect_peeling_supportable) (vect_analyze_data_ref_accesses, vect_prune_runtime_alias_test_list) (vect_analyze_data_refs): Likewise. (vect_slp_analyze_data_ref_dependence): Likewise. Take a vec_info argument. (vect_find_same_alignment_drs): Likewise. (vect_slp_analyze_node_dependences): Update calls accordingly. (vect_analyze_data_refs_alignment): Likewise. Use vec_info::lookup_dr instead of DR_VECT_AUX. (vect_get_peeling_costs_all_drs): Take a loop_vec_info instead of a vector data references. Use vec_info::lookup_dr instead of DR_VECT_AUX. (vect_peeling_hash_get_lowest_cost): Update calls accordingly. (vect_enhance_data_refs_alignment): Likewise. Use vec_info::lookup_dr instead of DR_VECT_AUX. From-SVN: r263155
2018-07-31[39/46] Change STMT_VINFO_UNALIGNED_DR to a dr_vec_infoRichard Sandiford1-2/+2
After previous changes, it makes more sense for STMT_VINFO_UNALIGNED_DR to be dr_vec_info rather than a data_reference. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (_loop_vec_info::unaligned_dr): Change to dr_vec_info. * tree-vect-data-refs.c (vect_enhance_data_refs_alignment): Update accordingly. * tree-vect-loop.c (vect_analyze_loop_2): Likewise. * tree-vect-loop-manip.c (get_misalign_in_elems): Likewise. (vect_gen_prolog_loop_niters): Likewise. From-SVN: r263154
2018-07-31[38/46] Use dr_vec_info to represent a data referenceRichard Sandiford1-8/+10
This patch makes various routines (mostly in tree-vect-data-refs.c) take dr_vec_infos rather than data_references. The affected routines are really dealing with the way that an access is going to vectorised, rather than with the original scalar access described by the data_reference. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (set_dr_misalignment, dr_misalignment) (DR_TARGET_ALIGNMENT, aligned_access_p, known_alignment_for_access_p) (vect_known_alignment_in_bytes, vect_dr_behavior) (vect_get_scalar_dr_size): Take references as dr_vec_infos instead of data_references. Update calls to other routines for which the same change has been made. * tree-vect-data-refs.c (vect_preserves_scalar_order_p): Take dr_vec_infos instead of stmt_vec_infos. (vect_analyze_data_ref_dependence): Update call accordingly. (vect_slp_analyze_data_ref_dependence) (vect_record_base_alignments): Use DR_VECT_AUX. (vect_calculate_target_alignment, vect_compute_data_ref_alignment) (vect_update_misalignment_for_peel, verify_data_ref_alignment) (vector_alignment_reachable_p, vect_get_data_access_cost) (vect_peeling_supportable, vect_analyze_group_access_1) (vect_analyze_group_access, vect_analyze_data_ref_access) (vect_vfa_segment_size, vect_vfa_access_size, vect_vfa_align) (vect_compile_time_alias, vect_small_gap_p) (vectorizable_with_step_bound_p, vect_duplicate_ssa_name_ptr_info): (vect_supportable_dr_alignment): Take references as dr_vec_infos instead of data_references. Update calls to other routines for which the same change has been made. (vect_verify_datarefs_alignment, vect_get_peeling_costs_all_drs) (vect_find_same_alignment_drs, vect_analyze_data_refs_alignment) (vect_slp_analyze_and_verify_node_alignment) (vect_analyze_data_ref_accesses, vect_prune_runtime_alias_test_list) (vect_create_addr_base_for_vector_ref, vect_create_data_ref_ptr) (vect_setup_realignment): Use dr_vec_infos. Update calls after above changes. (_vect_peel_info::dr): Replace with... (_vect_peel_info::dr_info): ...this new field. (vect_peeling_hash_get_most_frequent) (vect_peeling_hash_choose_best_peeling): Update accordingly. (vect_peeling_hash_get_lowest_cost): (vect_enhance_data_refs_alignment): Likewise. Update calls to other routines for which the same change has been made. (vect_peeling_hash_insert): Likewise. Take a dr_vec_info instead of a data_reference. * tree-vect-loop-manip.c (get_misalign_in_elems) (vect_gen_prolog_loop_niters): Use dr_vec_infos. Update calls after above changes. * tree-vect-loop.c (vect_analyze_loop_2): Likewise. * tree-vect-stmts.c (vect_get_store_cost, vect_get_load_cost) (vect_truncate_gather_scatter_offset, compare_step_with_zero) (get_group_load_store_type, get_negative_load_store_type) (vect_get_data_ptr_increment, vectorizable_store) (vectorizable_load): Likewise. (ensure_base_align): Take a dr_vec_info instead of a data_reference. Update calls to other routines for which the same change has been made. From-SVN: r263153
2018-07-31[31/46] Use stmt_vec_info in function interfaces (part 1)Richard Sandiford1-7/+8
This first (less mechanical) part handles cases that involve changes in the callers or non-trivial changes in the functions themselves. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vect-data-refs.c (vect_describe_gather_scatter_call): Take a stmt_vec_info instead of a gcall. (vect_check_gather_scatter): Update call accordingly. * tree-vect-loop-manip.c (iv_phi_p): Take a stmt_vec_info instead of a gphi. (vect_can_advance_ivs_p, vect_update_ivs_after_vectorizer) (slpeel_update_phi_nodes_for_loops):): Update calls accordingly. * tree-vect-loop.c (vect_transform_loop_stmt): Take a stmt_vec_info instead of a gimple stmt. (vect_transform_loop): Update calls accordingly. * tree-vect-slp.c (vect_split_slp_store_group): Take and return stmt_vec_infos instead of gimple stmts. (vect_analyze_slp_instance): Update use accordingly. * tree-vect-stmts.c (read_vector_array, write_vector_array) (vect_clobber_variable, vect_stmt_relevant_p, permute_vec_elements) (vect_use_strided_gather_scatters_p, vect_build_all_ones_mask) (vect_build_zero_merge_argument, vect_get_gather_scatter_ops) (vect_gen_widened_results_half, vect_get_loop_based_defs) (vect_create_vectorized_promotion_stmts, can_vectorize_live_stmts): Take a stmt_vec_info instead of a gimple stmt and pass stmt_vec_infos down to subroutines. From-SVN: r263146
2018-07-31[28/46] Use stmt_vec_info instead of gimple stmts internally (part 1)Richard Sandiford1-2/+2
This first part makes functions use stmt_vec_infos instead of gimple stmts in cases where the stmt_vec_info was already available and where the change is mechanical. Most of it is just replacing "stmt" with "stmt_info". 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vect-data-refs.c (vect_slp_analyze_node_dependences): (vect_check_gather_scatter, vect_create_data_ref_ptr, bump_vector_ptr) (vect_permute_store_chain, vect_setup_realignment) (vect_permute_load_chain, vect_shift_permute_load_chain) (vect_transform_grouped_load): Use stmt_vec_info rather than gimple stmts internally, and when passing values to other vectorizer routines. * tree-vect-loop-manip.c (vect_can_advance_ivs_p): Likewise. * tree-vect-loop.c (vect_analyze_scalar_cycles_1) (vect_analyze_loop_operations, get_initial_def_for_reduction) (vect_create_epilog_for_reduction, vectorize_fold_left_reduction) (vectorizable_reduction, vectorizable_induction) (vectorizable_live_operation, vect_transform_loop_stmt) (vect_transform_loop): Likewise. * tree-vect-patterns.c (vect_reassociating_reduction_p) (vect_recog_widen_op_pattern, vect_recog_mixed_size_cond_pattern) (vect_recog_bool_pattern, vect_recog_gather_scatter_pattern): Likewise. * tree-vect-slp.c (vect_analyze_slp_instance): Likewise. (vect_slp_analyze_node_operations_1): Likewise. * tree-vect-stmts.c (vect_mark_relevant, process_use) (exist_non_indexing_operands_for_use_p, vect_init_vector_1) (vect_mark_stmts_to_be_vectorized, vect_get_vec_def_for_operand) (vect_finish_stmt_generation_1, get_group_load_store_type) (get_load_store_type, vect_build_gather_load_calls) (vectorizable_bswap, vectorizable_call, vectorizable_simd_clone_call) (vect_create_vectorized_demotion_stmts, vectorizable_conversion) (vectorizable_assignment, vectorizable_shift, vectorizable_operation) (vectorizable_store, vectorizable_load, vectorizable_condition) (vectorizable_comparison, vect_analyze_stmt, vect_transform_stmt) (supportable_widening_operation): Likewise. (vect_get_vector_types_for_stmt): Likewise. * tree-vectorizer.h (vect_dr_behavior): Likewise. From-SVN: r263143
2018-07-31[27/46] Remove duplicated stmt_vec_info lookupsRichard Sandiford1-3/+4
Various places called vect_dr_stmt or vinfo_for_stmt multiple times on the same input. This patch makes them reuse the earlier result. It also splits a couple of single vinfo_for_stmt calls out into separate statements so that they can be reused in later patches. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vect-data-refs.c (vect_analyze_data_ref_dependence) (vect_slp_analyze_node_dependences, vect_analyze_data_ref_accesses) (vect_permute_store_chain, vect_permute_load_chain) (vect_shift_permute_load_chain, vect_transform_grouped_load): Avoid repeated stmt_vec_info lookups. * tree-vect-loop-manip.c (vect_can_advance_ivs_p): Likewise. (vect_update_ivs_after_vectorizer): Likewise. * tree-vect-loop.c (vect_is_simple_reduction): Likewise. (vect_create_epilog_for_reduction, vectorizable_reduction): Likewise. * tree-vect-patterns.c (adjust_bool_stmts): Likewise. * tree-vect-slp.c (vect_analyze_slp_instance): Likewise. (vect_bb_slp_scalar_cost): Likewise. * tree-vect-stmts.c (get_group_alias_ptr_type): Likewise. From-SVN: r263142
2018-07-31[23/46] Make LOOP_VINFO_MAY_MISALIGN_STMTS use stmt_vec_infoRichard Sandiford1-7/+6
This patch changes LOOP_VINFO_MAY_MISALIGN_STMTS from an auto_vec<gimple *> to an auto_vec<stmt_vec_info>. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (_loop_vec_info::may_misalign_stmts): Change from an auto_vec<gimple *> to an auto_vec<stmt_vec_info>. * tree-vect-data-refs.c (vect_enhance_data_refs_alignment): Update accordingly. * tree-vect-loop-manip.c (vect_create_cond_for_align_checks): Likewise. From-SVN: r263138
2018-07-31[19/46] Make vect_dr_stmt return a stmt_vec_infoRichard Sandiford1-5/+3
This patch makes vect_dr_stmt return a stmt_vec_info instead of a gimple stmt. Rather than retain a separate gimple stmt variable in cases where both existed, the patch replaces uses of the gimple variable with the uses of the stmt_vec_info. Later patches do this more generally. Many things that are keyed off a data_reference would these days be better keyed off a stmt_vec_info, but it's more convenient to do that later in the series. The vect_dr_size calls that are left over do still benefit from this patch. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (vect_dr_stmt): Return a stmt_vec_info rather than a gimple stmt. * tree-vect-data-refs.c (vect_analyze_data_ref_dependence) (vect_slp_analyze_data_ref_dependence, vect_record_base_alignments) (vect_calculate_target_alignmentm, vect_compute_data_ref_alignment) (vect_update_misalignment_for_peel, vect_verify_datarefs_alignment) (vector_alignment_reachable_p, vect_get_data_access_cost) (vect_get_peeling_costs_all_drs, vect_peeling_hash_get_lowest_cost) (vect_peeling_supportable, vect_enhance_data_refs_alignment) (vect_find_same_alignment_drs, vect_analyze_data_refs_alignment) (vect_analyze_group_access_1, vect_analyze_group_access) (vect_analyze_data_ref_access, vect_analyze_data_ref_accesses) (vect_vfa_access_size, vect_small_gap_p, vect_analyze_data_refs) (vect_supportable_dr_alignment): Remove vinfo_for_stmt from the result of vect_dr_stmt and use the stmt_vec_info instead of the associated gimple stmt. * tree-vect-loop-manip.c (get_misalign_in_elems): Likewise. (vect_gen_prolog_loop_niters): Likewise. * tree-vect-loop.c (vect_analyze_loop_2): Likewise. From-SVN: r263134
2018-07-31[10/46] Temporarily make stmt_vec_info a classRichard Sandiford1-1/+1
This patch turns stmt_vec_info into an unspeakably bad wrapper class and adds an implicit conversion to the associated gimple stmt. Having this conversion makes the rest of the series easier to write, but since the class goes away again at the end of the series, I've not bothered adding any comments or tried to make it pretty. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (stmt_vec_info): Temporarily change from a typedef to a wrapper class. (NULL_STMT_VEC_INFO): New macro. (vec_info::stmt_infos): Change to vec<stmt_vec_info>. (stmt_vec_info::operator*): New function. (stmt_vec_info::operator gimple *): Likewise. (set_vinfo_for_stmt): Use NULL_STMT_VEC_INFO. (add_stmt_costs): Likewise. * tree-vect-loop-manip.c (iv_phi_p): Likewise. * tree-vect-loop.c (vect_compute_single_scalar_iteration_cost) (vect_get_known_peeling_cost): Likewise. (vect_estimate_min_profitable_iters): Likewise. * tree-vect-patterns.c (vect_init_pattern_stmt): Likewise. * tree-vect-slp.c (vect_remove_slp_scalar_calls): Likewise. * tree-vect-stmts.c (vect_build_gather_load_calls): Likewise. (vectorizable_store, free_stmt_vec_infos): Likewise. (new_stmt_vec_info): Change return type of xcalloc to _stmt_vec_info *. From-SVN: r263125