aboutsummaryrefslogtreecommitdiff
path: root/gcc/tree-vectorizer.h
AgeCommit message (Collapse)AuthorFilesLines
2019-01-01Update copyright years.Jakub Jelinek1-1/+1
From-SVN: r267494
2018-11-15re PR tree-optimization/88031 (ICE in vectorizable_reduction, at ↵Richard Biener1-1/+1
tree-vect-loop.c:6953) 2018-11-15 Richard Biener <rguenther@suse.de> PR tree-optimization/88031 * tree-vect-loop.c (vectorizable_reduction): Move check for multiple types earlier so we get the expected dump. Simplify calls to vectorizable_condition. * tree-vect-stmts.h (vectorizable_condition): Update prototype. * tree-vect-stmts.c (vectorizable_condition): Instead of reduc_def and reduc_index take just a flag. Simplify code-generation now that we can rely on the defs being set up. (vectorizable_comparison): Remove unused argument. * gcc.dg/pr88031.c: New testcase. From-SVN: r266182
2018-11-13[PATCH][GCC] Make DR_TARGET_ALIGNMENT compile time variableAndre Vieira1-3/+3
This patch enables targets to describe DR_TARGET_ALIGNMENT as a compile-time variable. It does so by turning the variable into a 'poly_uint64'. gcc/ChangeLog: 2018-11-13 Andre Vieira <andre.simoesdiasvieira@arm.com> * config/aarch64/aarch64.c (aarch64_vectorize_preferred_vector_alignment): Change return type to poly_uint64. (aarch64_simd_vector_alignment_reachable): Adapt to preferred vector alignment being a poly int. * doc/tm.texi (TARGET_VECTORIZE_PREFERRED_VECTOR_ALIGNMENT): Change return type to poly_uint64. * target.def (default_preferred_vector_alignment): Likewise. * targhooks.c (default_preferred_vector_alignment): Likewise. * targhooks.h (default_preferred_vector_alignment): Likewise. * tree-vect-data-refs.c (vect_calculate_target_alignment): Likewise. (vect_compute_data_ref_alignment): Adapt to vector alignment being a poly int. (vect_update_misalignment_for_peel): Likewise. (vect_enhance_data_refs_alignment): Likewise. (vect_find_same_alignment_drs): Likewise. (vect_duplicate_ssa_name_ptr_info): Likewise. (vect_setup_realignment): Likewise. (vect_can_force_dr_alignment_p): Change alignment parameter type to poly_uint64. * tree-vect-loop-manip.c (get_misalign_in_elems): Learn to construct a mask with a compile time variable vector alignment. (vect_gen_prolog_loop_niters): Adapt to vector alignment being a poly int. (vect_do_peeling): Exit early if vector alignment is not constant. * tree-vect-stmts.c (ensure_base_align): Adapt to vector alignment being a poly int. (vectorizable_store): Likewise. (vectorizable_load): Likweise. * tree-vectorizer.h (struct dr_vec_info): Make target_alignment field a poly_uint64. (vect_known_alignment_in_bytes): Adapt to vector alignment being a poly int. (vect_can_force_dr_alignment_p): Change alignment parameter type to poly_uint64. From-SVN: r266072
2018-11-07re PR tree-optimization/87914 (gcc fails to vectorize bitreverse code)Richard Biener1-0/+3
2018-11-07 Richard Biener <rguenther@suse.de> PR tree-optimization/87914 * tree-vect-loop.c (vect_is_simple_reduction): Improve detection of nested cycles. (vectorizable_reduction): Handle shifts and rotates by dispatching to vectorizable_shift. * tree-vect-stmts.c (vect_get_vec_def_for_operand_1): Handle in-loop uses of vect_nested_cycle defs. Merge cycle and internal def cases. (vectorizable_shift): Export and handle being called as vect_nested_cycle. (vect_analyze_stmt): Call vectorizable_shift after vectorizable_reduction. * tree-vectorizer.h (vectorizable_shift): Declare. * lib/target-supports.exp (check_effective_target_vect_var_shift): New. (check_avx2_available): Likewise. * g++.dg/vect/pr87914.cc: New testcase. From-SVN: r265876
2018-10-26re PR tree-optimization/87105 (Autovectorization [X86, SSE2, AVX2, ↵Richard Biener1-0/+2
DoublePrecision]) 2018-10-26 Richard Biener <rguenther@suse.de> PR tree-optimization/87105 * tree-vectorizer.h (_slp_tree::refcnt): New member. * tree-vect-slp.c (vect_free_slp_tree): Decrement and honor refcnt. (vect_create_new_slp_node): Initialize refcnt to one. (bst_traits): Move. (scalar_stmts_set_t, bst_fail): Remove. (vect_build_slp_tree_2): Add bst_map argument and adjust calls. (vect_build_slp_tree): Add bst_map argument and lookup already created SLP nodes. (vect_print_slp_tree): Handle a SLP graph, print SLP node addresses. (vect_slp_rearrange_stmts): Handle a SLP graph. (vect_analyze_slp_instance): Adjust and free SLP nodes from the CSE map. Fix indenting. (vect_schedule_slp_instance): Add short-cut. * g++.dg/vect/slp-pr87105.cc: Adjust. * gcc.dg/torture/20181024-1.c: New testcase. * g++.dg/opt/20181025-1.C: Likewise. From-SVN: r265522
2018-10-25re PR tree-optimization/87665 (gcc HEAD (svn: 265340) breaks elements on resize)Richard Biener1-32/+12
2018-10-25 Richard Biener <rguenther@suse.de> PR tree-optimization/87665 PR tree-optimization/87745 * tree-vectorizer.h (get_earlier_stmt): Remove. (get_later_stmt): Pick up UID from the original non-pattern stmt. * gfortran.dg/20181025-1.f: New testcase. From-SVN: r265481
2018-10-08tree-vect-loop.c (vect_compute_single_scalar_iteration_cost): Open a dump scope.Richard Biener1-4/+7
2018-10-08 Richard Biener <rguenther@suse.de> * tree-vect-loop.c (vect_compute_single_scalar_iteration_cost): Open a dump scope. * tree-vectorizer.c (dump_stmt_cost): Add cost param and dump it. * tree-vectorizer.h (dump_stmt_cost): Adjust. (add_stmt_cost): Dump return value of the hook. From-SVN: r264923
2018-10-04Report vectorization problems via a new opt_problem classDavid Malcolm1-18/+27
This is v3 of the patch; previous versions were: v2: https://gcc.gnu.org/ml/gcc-patches/2018-07/msg00446.html v1: https://gcc.gnu.org/ml/gcc-patches/2018-06/msg01462.html This patch introduces a class opt_problem, along with wrapper classes for bool (opt_result) and for pointers (e.g. opt_loop_vec_info for loop_vec_info). opt_problem instances are created when an optimization problem is encountered, but only if dump_enabled_p. They are manually propagated up the callstack, and are manually reported at the "top level" of an optimization if dumping is enabled, to give the user a concise summary of the problem *after* the failure is reported. In particular, the location of the problematic statement is captured and emitted, rather than just the loop's location. For example: no-vfa-vect-102.c:24:3: missed: couldn't vectorize loop no-vfa-vect-102.c:27:7: missed: statement clobbers memory: __asm__ __volatile__("" : : : "memory"); Changed in v3: * This version bootstraps and passes regression testing (on x86_64-pc-linux-gnu). * added selftests, to exercise the opt_problem machinery * removed the "bool to opt_result" ctor, so that attempts to use e.g. return a bool from an opt_result-returning function will fail at compile time * use formatted printing within opt_problem ctor to replace the various dump_printf_loc calls * dropped i18n * changed the sense of vect_analyze_data_ref_dependence's return value (see the ChangeLog) * add MSG_PRIORITY_REEMITTED, so that -fopt-info can show the messages, without them messing up the counts in scan-tree-dump-times in DejaGnu tests gcc/ChangeLog: * Makefile.in (OBJS): Add opt-problem.o. * dump-context.h: Include "selftest.h. (selftest::temp_dump_context): New forward decl. (class dump_context): Make friend of class selftest::temp_dump_context. (dump_context::dump_loc_immediate): New decl. (class dump_pretty_printer): Move here from dumpfile.c. (class temp_dump_context): Move to namespace selftest. (temp_dump_context::temp_dump_context): Add param "forcibly_enable_dumping". (selftest::verify_dumped_text): (ASSERT_DUMPED_TEXT_EQ): Move here from dumpfile.c. (selftest::verify_item): (ASSERT_IS_TEXT): Move here from dumpfile.c. (ASSERT_IS_TREE): Likewise. (ASSERT_IS_GIMPLE): Likewise. * dumpfile.c (dump_context::dump_loc): Move immediate dumping to... (dump_context::dump_loc_immediate): ...this new function. (class dump_pretty_printer): Move to dump-context.h. (dump_switch_p_1): Don't enable MSG_PRIORITY_REEMITTED. (opt_info_switch_p_1): Enable MSG_PRIORITY_REEMITTED. (temp_dump_context::temp_dump_context): Move to "selftest" namespace. Add param "forcibly_enable_dumping", and use it to conditionalize the use of m_pp; (selftest::verify_dumped_text): Make non-static. (ASSERT_DUMPED_TEXT_EQ): Move to dump-context.h. (selftest::verify_item): Make non-static. (ASSERT_IS_TEXT): Move to dump-context.h. (ASSERT_IS_TREE): Likewise. (ASSERT_IS_GIMPLE): Likewise. (selftest::test_capture_of_dump_calls): Pass "true" for new param of temp_dump_context. * dumpfile.h (enum dump_flag): Add MSG_PRIORITY_REEMITTED, adding it to MSG_ALL_PRIORITIES. Update values of TDF_COMPARE_DEBUG and TDF_COMPARE_DEBUG. * opt-problem.cc: New file. * opt-problem.h: New file. * optinfo-emit-json.cc (selftest::test_building_json_from_dump_calls): Pass "true" for new param of temp_dump_context. * optinfo.cc (optinfo_kind_to_dump_flag): New function. (optinfo::emit_for_opt_problem): New function. (optinfo::emit): Clarity which emit_item is used. * optinfo.h (optinfo::get_dump_location): New accessor. (optinfo::emit_for_opt_problem): New decl. (optinfo::emit): Make const. * selftest-run-tests.c (selftest::run_tests): Call selftest::opt_problem_cc_tests. * selftest.h (selftest::opt_problem_cc_tests): New decl. * tree-data-ref.c (dr_analyze_innermost): Convert return type from bool to opt_result, converting fprintf messages to opt_result::failure_at calls. Add "stmt" param for use by the failure_at calls. (create_data_ref): Pass "stmt" to the dr_analyze_innermost call. (runtime_alias_check_p): Convert return type from bool to opt_result, converting dump_printf calls to opt_result::failure_at, using the statement DDR_A for their location. (find_data_references_in_stmt): Convert return type from bool to opt_result, converting "return false" to opt_result::failure_at with a new message. * tree-data-ref.h: Include "opt-problem.h". (dr_analyze_innermost): Convert return type from bool to opt_result, and add a const gimple * param. (find_data_references_in_stmt): Convert return type from bool to opt_result. (runtime_alias_check_p): Likewise. * tree-predcom.c (find_looparound_phi): Pass "init_stmt" to dr_analyze_innermost. * tree-vect-data-refs.c (vect_mark_for_runtime_alias_test): Convert return type from bool to opt_result, adding a message for the PARAM_VECT_MAX_VERSION_FOR_ALIAS_CHECKS zero case. (vect_analyze_data_ref_dependence): Convert return type from bool to opt_result. Change sense of return type from "false" effectively meaning "no problems" to "false" meaning a problem, so that "return false" becomes "return opt_result::success". Convert "return true" calls to opt_result::failure_at, using the location of statement A rather than vect_location. (vect_analyze_data_ref_dependences): Convert return type from bool to opt_result. (verify_data_ref_alignment): Likewise, converting dump_printf_loc calls to opt_result::failure_at, using the stmt location rather than vect_location. (vect_verify_datarefs_alignment): Convert return type from bool to opt_result. (vect_enhance_data_refs_alignment): Likewise. Split local "stat" into multiple more-tightly-scoped copies. (vect_analyze_data_refs_alignment): Convert return type from bool to opt_result. (vect_analyze_data_ref_accesses): Likewise, converting a "return false" to a "return opt_result::failure_at", adding a new message. (vect_prune_runtime_alias_test_list): Convert return type from bool to opt_result, converting dump_printf_loc to opt_result::failure_at. Add a %G to show the pertinent statement, and use the stmt's location rather than vect_location. (vect_find_stmt_data_reference): Convert return type from bool to opt_result, converting dump_printf_loc to opt_result::failure_at, using stmt's location. (vect_analyze_data_refs): Convert return type from bool to opt_result. Convert "return false" to "return opt_result::failure_at", adding messages as needed. * tree-vect-loop.c (vect_determine_vf_for_stmt_1): Convert return type from bool to opt_result. (vect_determine_vf_for_stmt): Likewise. (vect_determine_vectorization_factor): Likewise, converting dump_printf_loc to opt_result::failure_at, using location of phi rather than vect_location. (vect_analyze_loop_form_1): Convert return type from bool to opt_result, converting dump_printf_loc calls, retaining the use of vect_location. (vect_analyze_loop_form): Convert return type from loop_vec_info to opt_loop_vec_info. (vect_analyze_loop_operations): Convert return type from bool to opt_result, converting dump_printf_loc calls, using the location of phi/stmt rather than vect_location where available. Convert various "return false" to "return opt_result::failure_at" with "unsupported phi" messages. (vect_get_datarefs_in_loop): Convert return type from bool to opt_result. Add a message for the PARAM_LOOP_MAX_DATAREFS_FOR_DATADEPS failure. (vect_analyze_loop_2): Convert return type from bool to opt_result. Ensure "ok" is set to a opt_result::failure_at before each "goto again;", adding new messages where needed. Add "unsupported grouped {store|load}" messages. (vect_analyze_loop): Convert return type from loop_vec_info to opt_loop_vec_info. * tree-vect-slp.c (vect_analyze_slp): Convert return type from bool to opt_result. * tree-vect-stmts.c (process_use): Likewise, converting dump_printf_loc call and using stmt location, rather than vect_location. (vect_mark_stmts_to_be_vectorized): Likeise. (vect_analyze_stmt): Likewise, adding a %G. (vect_get_vector_types_for_stmt): Convert return type from bool to opt_result, converting dump_printf_loc calls and using stmt location, rather than vect_location. (vect_get_mask_type_for_stmt): Convert return type from tree to opt_tree, converting dump_printf_loc calls and using stmt location. * tree-vectorizer.c: Include "opt-problem.h. (try_vectorize_loop_1): Flag "Analyzing loop at" dump message as MSG_PRIORITY_INTERNALS. Convert local "loop_vinfo" from loop_vec_info to opt_loop_vec_info. If if fails, and dumping is enabled, use it to report at the top level "couldn't vectorize loop" followed by the problem. * tree-vectorizer.h (opt_loop_vec_info): New typedef. (vect_mark_stmts_to_be_vectorized): Convert return type from bool to opt_result. (vect_analyze_stmt): Likewise. (vect_get_vector_types_for_stmt): Likewise. (tree vect_get_mask_type_for_stmt): Likewise. (vect_analyze_data_ref_dependences): Likewise. (vect_enhance_data_refs_alignment): Likewise. (vect_analyze_data_refs_alignment): Likewise. (vect_verify_datarefs_alignment): Likewise. (vect_analyze_data_ref_accesses): Likewise. (vect_prune_runtime_alias_test_list): Likewise. (vect_find_stmt_data_reference): Likewise. (vect_analyze_data_refs): Likewise. (vect_analyze_loop): Convert return type from loop_vec_info to opt_loop_vec_info. (vect_analyze_loop_form): Likewise. (vect_analyze_slp): Convert return type from bool to opt_result. gcc/testsuite/ChangeLog: * gcc.dg/vect/nodump-vect-opt-info-2.c: New test. * gcc.dg/vect/vect-alias-check-4.c: Add "-fopt-info-vec-all" to dg-additional-options. Add dg-message and dg-missed directives to verify that -fopt-info messages are written at the correct locations. From-SVN: r264852
2018-10-04Add -fopt-info-internalsDavid Malcolm1-1/+5
This patch introduces a verbosity level to dump messages: "user-facing" vs "internals". By default, messages at the top-level dump scope are "user-facing", whereas those that are in nested scopes are implicitly "internals", and are filtered out by -fopt-info unless a new "-internals" sub-option of "-fopt-info" is supplied (intended purely for use by GCC developers). Dumpfiles are unaffected by the change. Given that the vectorizer is the only subsystem using AUTO_DUMP_SCOPE (via DUMP_VECT_SCOPE), this only affects the vectorizer. Filtering out these implementation-detail messages goes a long way towards making -fopt-info-vec-all more accessible to advanced end-users; the follow-up patch restores the most pertinent missing details. gcc/ChangeLog: * doc/invoke.texi (-fopt-info): Document new "internals" sub-option. * dump-context.h (dump_context::apply_dump_filter_p): New decl. * dumpfile.c (dump_options): Update for renaming of MSG_ALL to MSG_ALL_KINDS. (optinfo_verbosity_options): Add "internals". (kind_as_string): Update for renaming of MSG_ALL to MSG_ALL_KINDS. (dump_context::apply_dump_filter_p): New member function. (dump_context::dump_loc): Use apply_dump_filter_p rather than explicitly masking the dump_kind. (dump_context::begin_scope): Increment the scope depth first. Use apply_dump_filter_p rather than explicitly masking the dump_kind. (dump_context::emit_item): Use apply_dump_filter_p rather than explicitly masking the dump_kind. (dump_dec): Likewise. (dump_hex): Likewise. (dump_switch_p_1): Default to MSG_ALL_PRIORITIES. (opt_info_switch_p_1): Default to MSG_PRIORITY_USER_FACING. (opt_info_switch_p): Update handling of default MSG_OPTIMIZED_LOCATIONS to cope with default of MSG_PRIORITY_USER_FACING. (dump_basic_block): Use apply_dump_filter_p rather than explicitly masking the dump_kind. (selftest::test_capture_of_dump_calls): Update test_dump_context instances to use MSG_ALL_KINDS | MSG_PRIORITY_USER_FACING rather than MSG_ALL. Generalize scope test to be run at all four combinations of with/without MSG_PRIORITY_USER_FACING and MSG_PRIORITY_INTERNALS, adding examples of explicit priority for each of the two values. * dumpfile.h (enum dump_flag): Add comment about the MSG_* flags. Rename MSG_ALL to MSG_ALL_KINDS. Add MSG_PRIORITY_USER_FACING, MSG_PRIORITY_INTERNALS, and MSG_ALL_PRIORITIES, updating the values for TDF_COMPARE_DEBUG and TDF_ALL_VALUES. (AUTO_DUMP_SCOPE): Add a note to the comment about the interaction with MSG_PRIORITY_*. * tree-vect-loop-manip.c (vect_loop_versioning): Mark versioning dump messages as MSG_PRIORITY_USER_FACING. * tree-vectorizer.h (DUMP_VECT_SCOPE): Add a note to the comment about the interaction with MSG_PRIORITY_*. gcc/testsuite/ChangeLog: * gcc.dg/plugin/dump-1.c: Update expected output for test_scopes due to "-internals" not being selected. * gcc.dg/plugin/dump-2.c: New test, based on dump-1.c, with "-internals" added to re-enable the output from test_scopes. * gcc.dg/plugin/plugin.exp (plugin_test_list): Add dump-2.c. From-SVN: r264851
2018-08-01[06/11] Handle VMAT_INVARIANT separatelyRichard Sandiford1-1/+1
Invariant loads were handled as a variation on the code for contiguous loads. We detected whether they were invariant or not as a byproduct of creating the vector pointer ivs: vect_create_data_ref_ptr passed back an inv_p to say whether the pointer was invariant. But vectorised invariant loads just keep the original scalar load, so this meant that detecting invariant loads had the side-effect of creating an unwanted vector pointer iv. The placement of the code also meant that we'd create a vector load and then not use the result. In principle this is wrong code, since there's no guarantee that there's a vector's worth of accessible data at that address, but we rely on DCE to get rid of the load before any harm is done. E.g., for an invariant load in an inner loop (which seems like the more common use case for this code), we'd create: vectp_a.6_52 = &a + 4; # vectp_a.5_53 = PHI <vectp_a.5_54(9), vectp_a.6_52(2)> # vectp_a.5_55 = PHI <vectp_a.5_53(3), vectp_a.5_56(10)> vect_next_a_11.7_57 = MEM[(int *)vectp_a.5_55]; next_a_11 = a[_1]; vect_cst__58 = {next_a_11, next_a_11, next_a_11, next_a_11}; vectp_a.5_56 = vectp_a.5_55 + 4; vectp_a.5_54 = vectp_a.5_53 + 0; whereas all we want is: next_a_11 = a[_1]; vect_cst__58 = {next_a_11, next_a_11, next_a_11, next_a_11}; This patch moves the handling to its own block and makes vect_create_data_ref_ptr assert (when creating a full iv) that the address isn't invariant. The ncopies handling is unfortunate, but a preexisting issue. Richi's suggestion of using a vector of vector statements would let us reuse one statement for all copies. 2018-08-01 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (vect_create_data_ref_ptr): Remove inv_p parameter. * tree-vect-data-refs.c (vect_create_data_ref_ptr): Likewise. When creating an iv, assert that the step is not known to be zero. (vect_setup_realignment): Update call accordingly. * tree-vect-stmts.c (vectorizable_store): Likewise. (vectorizable_load): Likewise. Handle VMAT_INVARIANT separately. From-SVN: r263220
2018-08-01[05/11] Add a vect_stmt_to_vectorize helper functionRichard Sandiford1-0/+11
This patch adds a helper that does the opposite of vect_orig_stmt: go from the original scalar statement to the statement that should actually be vectorised. The use in the last two hunks of vectorizable_reduction are because reduc_stmt_info (first hunk) and stmt_info (second hunk) are already pattern statements if appropriate. 2018-08-01 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (vect_stmt_to_vectorize): New function. * tree-vect-loop.c (vect_update_vf_for_slp): Use it. (vectorizable_reduction): Likewise. * tree-vect-slp.c (vect_analyze_slp_instance): Likewise. (vect_detect_hybrid_slp_stmts): Likewise. * tree-vect-stmts.c (vect_is_simple_use): Likewise. From-SVN: r263219
2018-08-01[04/11] Add a vect_orig_stmt helper functionRichard Sandiford1-0/+11
This patch just adds a helper function for going from a potential pattern statement to the original scalar statement. 2018-08-01 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (vect_orig_stmt): New function. * tree-vect-data-refs.c (vect_preserves_scalar_order_p): Use it. * tree-vect-loop.c (vect_model_reduction_cost): Likewise. (vect_create_epilog_for_reduction): Likewise. (vectorizable_live_operation): Likewise. * tree-vect-slp.c (vect_find_last_scalar_stmt_in_slp): Likewise. (vect_detect_hybrid_slp_stmts, vect_schedule_slp): Likewise. * tree-vect-stmts.c (vectorizable_call): Likewise. (vectorizable_simd_clone_call, vect_remove_stores): Likewise. From-SVN: r263217
2018-08-01[03/11] Remove vect_transform_stmt grouped_store argumentRichard Sandiford1-1/+1
Nothing now uses the grouped_store value passed back by vect_transform_stmt, so we might as well remove it. 2018-08-01 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (vect_transform_stmt): Remove grouped_store argument. * tree-vect-stmts.c (vect_transform_stmt): Likewise. * tree-vect-loop.c (vect_transform_loop_stmt): Update call accordingly. (vect_transform_loop): Likewise. * tree-vect-slp.c (vect_schedule_slp_instance): Likewise. From-SVN: r263216
2018-08-01[02/11] Remove vect_schedule_slp return valueRichard Sandiford1-1/+1
Nothing now uses the vect_schedule_slp return value, so it's not worth propagating the value through vect_schedule_slp_instance. 2018-08-01 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (vect_schedule_slp): Return void. * tree-vect-slp.c (vect_schedule_slp_instance): Likewise. (vect_schedule_slp): Likewise. From-SVN: r263215
2018-07-31[46/46] Turn stmt_vec_info back into a typedefRichard Sandiford1-31/+1
This patch removes the stmt_vec_info wrapper class added near the beginning of the series and turns stmt_vec_info back into a typedef. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (stmt_vec_info): Turn back into a typedef. (NULL_STMT_VEC_INFO): Delete. (stmt_vec_info::operator*): Likewise. (stmt_vec_info::operator gimple *): Likewise. * tree-vect-loop.c (vectorizable_reduction): Use NULL instead of NULL_STMT_VEC_INFO. * tree-vect-patterns.c (vect_init_pattern_stmt): Likewise. (vect_reassociating_reduction_p): Likewise. * tree-vect-stmts.c (vect_build_gather_load_calls): Likewise. (vectorizable_store): Likewise. * tree-vectorizer.c (vec_info::set_vinfo_for_stmt): Likewise. (vec_info::free_stmt_vec_infos): Likewise. From-SVN: r263161
2018-07-31[45/46] Remove vect_stmt_in_region_pRichard Sandiford1-1/+0
Unlike the old vinfo_for_stmt, vec_info::lookup_stmt can cope with any statement, so there's no need to check beforehand that the statement is part of the vectorisable region. This means that there are no longer any calls to vect_stmt_in_region_p. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (vect_stmt_in_region_p): Delete. * tree-vectorizer.c (vect_stmt_in_region_p): Likewise. From-SVN: r263160
2018-07-31[44/46] Remove global vinfo_for_stmt-related routinesRichard Sandiford1-39/+6
There are no more direct uses of: - new_stmt_vec_info - set_vinfo_for_stmt - free_stmt_vec_infos - free_stmt_vec_info outside of vec_info, so they can now be private member functions. It also seemed better to put them in tree-vectorizer.c, along with the other vec_info routines. We can also get rid of: - vinfo_for_stmt - stmt_vec_info_vec - set_stmt_vec_info_vec since nothing now uses them. This was the main goal of the series. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (vec_info::new_vinfo_for_stmt) (vec_info::set_vinfo_for_stmt, vec_info::free_stmt_vec_infos) (vec_info::free_stmt_vec_info): New private member functions. (set_stmt_vec_info_vec, free_stmt_vec_infos, vinfo_for_stmt) (set_vinfo_for_stmt, new_stmt_vec_info, free_stmt_vec_info): Delete. * tree-parloops.c (gather_scalar_reductions): Remove calls to set_stmt_vec_info_vec and free_stmt_vec_infos. * tree-vect-loop.c (_loop_vec_info): Remove call to set_stmt_vec_info_vec. * tree-vect-stmts.c (new_stmt_vec_info, set_stmt_vec_info_vec) (free_stmt_vec_infos, free_stmt_vec_info): Delete in favor of... * tree-vectorizer.c (vec_info::new_stmt_vec_info) (vec_info::set_vinfo_for_stmt, vec_info::free_stmt_vec_infos) (vec_info::free_stmt_vec_info): ...these new functions. Remove assignments in {vec_info::,}new_stmt_vec_info that are redundant with the clearing in the xcalloc. (stmt_vec_info_vec): Delete. (vec_info::vec_info): Don't call set_stmt_vec_info_vec. (vectorize_loops): Likewise. (vec_info::~vec_info): Remove argument from call to free_stmt_vec_infos. (vec_info::add_stmt): Remove vinfo argument from call to new_stmt_vec_info. From-SVN: r263159
2018-07-31[43/46] Make free_stmt_vec_info take a stmt_vec_infoRichard Sandiford1-1/+1
This patch makes free_stmt_vec_info take the stmt_vec_info that it's supposed to free and makes it free only that stmt_vec_info. Callers need to update the statement mapping where necessary (but now there are only a couple of callers). This in turns means that we can leave ~vec_info to do the actual freeing, since there's no longer a need to do it before resetting the gimple_uids. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (free_stmt_vec_info): Take a stmt_vec_info rather than a gimple stmt. * tree-vect-stmts.c (free_stmt_vec_info): Likewise. Don't free information for pattern statements when passed the original statement; instead wait to be passed the pattern statement itself. Don't call set_vinfo_for_stmt here. (free_stmt_vec_infos): Update call to free_stmt_vec_info. * tree-vect-loop.c (_loop_vec_info::~loop_vec_info): Don't free stmt_vec_infos here. * tree-vect-slp.c (_bb_vec_info::~bb_vec_info): Likewise. * tree-vectorizer.c (vec_info::remove_stmt): Nullify the statement's stmt_vec_infos entry. From-SVN: r263158
2018-07-31[42/46] Add vec_info::replace_stmtRichard Sandiford1-0/+1
This patch adds a helper for replacing a stmt_vec_info's statement with a new statement. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (vec_info::replace_stmt): Declare. * tree-vectorizer.c (vec_info::replace_stmt): New function. * tree-vect-slp.c (vect_remove_slp_scalar_calls): Use it. * tree-vect-stmts.c (vectorizable_call): Likewise. (vectorizable_simd_clone_call): Likewise. From-SVN: r263157
2018-07-31[41/46] Add vec_info::remove_stmtRichard Sandiford1-0/+1
This patch adds a new helper function for permanently removing a statement and its associated stmt_vec_info. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (vec_info::remove_stmt): Declare. * tree-vectorizer.c (vec_info::remove_stmt): New function. * tree-vect-loop-manip.c (vect_set_loop_condition): Use it. * tree-vect-loop.c (vect_transform_loop): Likewise. * tree-vect-slp.c (vect_schedule_slp): Likewise. * tree-vect-stmts.c (vect_remove_stores): Likewise. From-SVN: r263156
2018-07-31[40/46] Add vec_info::lookup_drRichard Sandiford1-16/+1
This patch replaces DR_VECT_AUX and vect_dr_stmt with a new vec_info::lookup_dr function, so that the lookup is relative to a particular vec_info rather than to global state. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (vec_info::lookup_dr): New member function. (vect_dr_stmt): Delete. * tree-vectorizer.c (vec_info::lookup_dr): New function. * tree-vect-loop-manip.c (vect_update_inits_of_drs): Use it instead of DR_VECT_AUX. * tree-vect-data-refs.c (vect_analyze_possibly_independent_ddr) (vect_analyze_data_ref_dependence, vect_record_base_alignments) (vect_verify_datarefs_alignment, vect_peeling_supportable) (vect_analyze_data_ref_accesses, vect_prune_runtime_alias_test_list) (vect_analyze_data_refs): Likewise. (vect_slp_analyze_data_ref_dependence): Likewise. Take a vec_info argument. (vect_find_same_alignment_drs): Likewise. (vect_slp_analyze_node_dependences): Update calls accordingly. (vect_analyze_data_refs_alignment): Likewise. Use vec_info::lookup_dr instead of DR_VECT_AUX. (vect_get_peeling_costs_all_drs): Take a loop_vec_info instead of a vector data references. Use vec_info::lookup_dr instead of DR_VECT_AUX. (vect_peeling_hash_get_lowest_cost): Update calls accordingly. (vect_enhance_data_refs_alignment): Likewise. Use vec_info::lookup_dr instead of DR_VECT_AUX. From-SVN: r263155
2018-07-31[39/46] Change STMT_VINFO_UNALIGNED_DR to a dr_vec_infoRichard Sandiford1-1/+1
After previous changes, it makes more sense for STMT_VINFO_UNALIGNED_DR to be dr_vec_info rather than a data_reference. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (_loop_vec_info::unaligned_dr): Change to dr_vec_info. * tree-vect-data-refs.c (vect_enhance_data_refs_alignment): Update accordingly. * tree-vect-loop.c (vect_analyze_loop_2): Likewise. * tree-vect-loop-manip.c (get_misalign_in_elems): Likewise. (vect_gen_prolog_loop_niters): Likewise. From-SVN: r263154
2018-07-31[38/46] Use dr_vec_info to represent a data referenceRichard Sandiford1-31/+30
This patch makes various routines (mostly in tree-vect-data-refs.c) take dr_vec_infos rather than data_references. The affected routines are really dealing with the way that an access is going to vectorised, rather than with the original scalar access described by the data_reference. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (set_dr_misalignment, dr_misalignment) (DR_TARGET_ALIGNMENT, aligned_access_p, known_alignment_for_access_p) (vect_known_alignment_in_bytes, vect_dr_behavior) (vect_get_scalar_dr_size): Take references as dr_vec_infos instead of data_references. Update calls to other routines for which the same change has been made. * tree-vect-data-refs.c (vect_preserves_scalar_order_p): Take dr_vec_infos instead of stmt_vec_infos. (vect_analyze_data_ref_dependence): Update call accordingly. (vect_slp_analyze_data_ref_dependence) (vect_record_base_alignments): Use DR_VECT_AUX. (vect_calculate_target_alignment, vect_compute_data_ref_alignment) (vect_update_misalignment_for_peel, verify_data_ref_alignment) (vector_alignment_reachable_p, vect_get_data_access_cost) (vect_peeling_supportable, vect_analyze_group_access_1) (vect_analyze_group_access, vect_analyze_data_ref_access) (vect_vfa_segment_size, vect_vfa_access_size, vect_vfa_align) (vect_compile_time_alias, vect_small_gap_p) (vectorizable_with_step_bound_p, vect_duplicate_ssa_name_ptr_info): (vect_supportable_dr_alignment): Take references as dr_vec_infos instead of data_references. Update calls to other routines for which the same change has been made. (vect_verify_datarefs_alignment, vect_get_peeling_costs_all_drs) (vect_find_same_alignment_drs, vect_analyze_data_refs_alignment) (vect_slp_analyze_and_verify_node_alignment) (vect_analyze_data_ref_accesses, vect_prune_runtime_alias_test_list) (vect_create_addr_base_for_vector_ref, vect_create_data_ref_ptr) (vect_setup_realignment): Use dr_vec_infos. Update calls after above changes. (_vect_peel_info::dr): Replace with... (_vect_peel_info::dr_info): ...this new field. (vect_peeling_hash_get_most_frequent) (vect_peeling_hash_choose_best_peeling): Update accordingly. (vect_peeling_hash_get_lowest_cost): (vect_enhance_data_refs_alignment): Likewise. Update calls to other routines for which the same change has been made. (vect_peeling_hash_insert): Likewise. Take a dr_vec_info instead of a data_reference. * tree-vect-loop-manip.c (get_misalign_in_elems) (vect_gen_prolog_loop_niters): Use dr_vec_infos. Update calls after above changes. * tree-vect-loop.c (vect_analyze_loop_2): Likewise. * tree-vect-stmts.c (vect_get_store_cost, vect_get_load_cost) (vect_truncate_gather_scatter_offset, compare_step_with_zero) (get_group_load_store_type, get_negative_load_store_type) (vect_get_data_ptr_increment, vectorizable_store) (vectorizable_load): Likewise. (ensure_base_align): Take a dr_vec_info instead of a data_reference. Update calls to other routines for which the same change has been made. From-SVN: r263153
2018-07-31[37/46] dr_aux tweaksRichard Sandiford1-37/+48
This patch makes dr_aux link back to both the scalar data_reference and the containing stmt_vec_info, so that it becomes a suitable key for a vectorisable reference. The data_reference link is just STMT_VINFO_DATA_REF, moved here from _stmt_vec_info. The stmt pointer is a new field and always tracks the current stmt_vec_info for the reference (which might be a pattern stmt or the original stmt). The patch also makes the dr_aux in this current stmt be the one that counts, rather than have the information stay with the DR_STMT. A new macro (STMT_VINFO_DR_INFO) gives this information for a given stmt_info. In future we could make dr_aux a separate structure, which would be useful if we want to support multiple data references per stmt. That seems too much of a diversion for this series though. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (vec_info::move_dr): New member function. (dataref_aux): Rename to... (dr_vec_info): ...this and add "dr" and "stmt" fields. (_stmt_vec_info::dr_aux): Update accordingly. (_stmt_vec_info::data_ref_info): Delete. (STMT_VINFO_GROUPED_ACCESS, DR_GROUP_FIRST_ELEMENT) (DR_GROUP_NEXT_ELEMENT, DR_GROUP_SIZE, DR_GROUP_STORE_COUNT) (DR_GROUP_GAP, DR_GROUP_SAME_DR_STMT, REDUC_GROUP_FIRST_ELEMENT): (REDUC_GROUP_NEXT_ELEMENT, REDUC_GROUP_SIZE): Use dr_aux.dr instead of data_ref. (STMT_VINFO_DATA_REF): Likewise. Turn into an lvalue. (STMT_VINFO_DR_INFO): New macro. (DR_VECT_AUX): Use STMT_VINFO_DR_INKFO and vect_dr_stmt. (set_dr_misalignment): Update after rename of dataref_aux. (vect_dr_stmt): Move earlier in file. Return dr_aux.stmt. * tree-vect-stmts.c (new_stmt_vec_info): Remove redundant initialization of STMT_VINFO_DATA_REF. * tree-vectorizer.c (vec_info::move_dr): New function. * tree-vect-patterns.c (vect_recog_bool_pattern) (vect_recog_mask_conversion_pattern) (vect_recog_gather_scatter_pattern): Use it. * tree-vect-data-refs.c (vect_analyze_data_refs): Initialize the "dr" and "stmt" fields of dr_vec_info instead of STMT_VINFO_DATA_REF. From-SVN: r263152
2018-07-31[36/46] Add a pattern_stmt_p field to stmt_vec_infoRichard Sandiford1-2/+7
This patch adds a pattern_stmt_p field to stmt_vec_info, so that it's possible to tell whether the statement is a pattern statement without referring to other statements. The new field goes in what was previously a hole in the structure, so the size is the same as before. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (_stmt_vec_info::pattern_stmt_p): New field. (is_pattern_stmt_p): Use it. * tree-vect-patterns.c (vect_init_pattern_stmt): Set pattern_stmt_p on pattern statements. From-SVN: r263151
2018-07-31[34/46] Alter interface to vect_get_vec_def_for_stmt_copyRichard Sandiford1-2/+2
This patch makes vect_get_vec_def_for_stmt_copy take a vec_info rather than a vect_def_type. If the vector operand passed in is defined in the vectorised region, we should look for copies in the normal way. If it's defined in an external statement (such as by vect_init_vector_1) we should just use the original value. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (vect_get_vec_defs_for_stmt_copy) (vect_get_vec_def_for_stmt_copy): Take a vec_info rather than a vect_def_type for the first argument. * tree-vect-stmts.c (vect_get_vec_defs_for_stmt_copy): Likewise. (vect_get_vec_def_for_stmt_copy): Likewise. Return the original operand if it isn't defined by a vectorized statement. (vect_build_gather_load_calls): Remove the mask_dt argument and update calls to vect_get_vec_def_for_stmt_copy. (vectorizable_bswap): Likewise the dt argument. (vectorizable_call): Update calls to vectorizable_bswap and vect_get_vec_def_for_stmt_copy. (vectorizable_simd_clone_call, vectorizable_assignment) (vectorizable_shift, vectorizable_operation, vectorizable_condition) (vectorizable_comparison): Update calls to vect_get_vec_def_for_stmt_copy. (vectorizable_store): Likewise. Remove now-unnecessary calls to vect_is_simple_use. (vect_get_loop_based_defs): Remove dt argument and update call to vect_get_vec_def_for_stmt_copy. (vectorizable_conversion): Update calls to vect_get_loop_based_defs and vect_get_vec_def_for_stmt_copy. (vectorizable_load): Update calls to vect_build_gather_load_calls and vect_get_vec_def_for_stmt_copy. * tree-vect-loop.c (vect_create_epilog_for_reduction) (vectorizable_reduction, vectorizable_live_operation): Update calls to vect_get_vec_def_for_stmt_copy. From-SVN: r263149
2018-07-31[32/46] Use stmt_vec_info in function interfaces (part 2)Richard Sandiford1-37/+37
This second part handles the mechanical change from a gimple stmt argument to a stmt_vec_info argument. It updates the function comments if they referred to the argument by name, but it doesn't try to retrofit mentions to other functions. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (nested_in_vect_loop_p): Move further down file and take a stmt_vec_info instead of a gimple stmt. (supportable_widening_operation, vect_finish_replace_stmt) (vect_finish_stmt_generation, vect_get_store_rhs) (vect_get_vec_def_for_operand_1, vect_get_vec_def_for_operand) (vect_get_vec_defs, vect_init_vector, vect_transform_stmt) (vect_remove_stores, vect_analyze_stmt, vectorizable_condition) (vect_get_smallest_scalar_type, vect_check_gather_scatter) (vect_create_data_ref_ptr, bump_vector_ptr) (vect_permute_store_chain, vect_setup_realignment) (vect_transform_grouped_load, vect_record_grouped_load_vectors) (vect_create_addr_base_for_vector_ref, vectorizable_live_operation) (vectorizable_reduction, vectorizable_induction) (get_initial_def_for_reduction, is_simple_and_all_uses_invariant) (vect_get_place_in_interleaving_chain): Take stmt_vec_infos rather than gimple stmts as arguments. * tree-vect-data-refs.c (vect_get_smallest_scalar_type) (vect_preserves_scalar_order_p, vect_slp_analyze_node_dependences) (can_group_stmts_p, vect_check_gather_scatter) (vect_create_addr_base_for_vector_ref, vect_create_data_ref_ptr) (bump_vector_ptr, vect_permute_store_chain, vect_setup_realignment) (vect_permute_load_chain, vect_shift_permute_load_chain) (vect_transform_grouped_load) (vect_record_grouped_load_vectors): Likewise. * tree-vect-loop.c (vect_fixup_reduc_chain) (get_initial_def_for_reduction, vect_create_epilog_for_reduction) (vectorize_fold_left_reduction, is_nonwrapping_integer_induction) (vectorizable_reduction, vectorizable_induction) (vectorizable_live_operation, vect_loop_kill_debug_uses): Likewise. * tree-vect-patterns.c (type_conversion_p, adjust_bool_stmts) (vect_get_load_store_mask): Likewise. * tree-vect-slp.c (vect_get_place_in_interleaving_chain) (vect_analyze_slp_instance, vect_mask_constant_operand_p): Likewise. * tree-vect-stmts.c (vect_mark_relevant) (is_simple_and_all_uses_invariant) (exist_non_indexing_operands_for_use_p, process_use) (vect_init_vector_1, vect_init_vector, vect_get_vec_def_for_operand_1) (vect_get_vec_def_for_operand, vect_get_vec_defs) (vect_finish_stmt_generation_1, vect_finish_replace_stmt) (vect_finish_stmt_generation, vect_truncate_gather_scatter_offset) (compare_step_with_zero, vect_get_store_rhs, get_group_load_store_type) (get_negative_load_store_type, get_load_store_type) (vect_check_load_store_mask, vect_check_store_rhs) (vect_build_gather_load_calls, vect_get_strided_load_store_ops) (vectorizable_bswap, vectorizable_call, vectorizable_simd_clone_call) (vect_create_vectorized_demotion_stmts, vectorizable_conversion) (vectorizable_assignment, vectorizable_shift, vectorizable_operation) (get_group_alias_ptr_type, vectorizable_store, hoist_defs_of_uses) (vectorizable_load, vectorizable_condition, vectorizable_comparison) (vect_analyze_stmt, vect_transform_stmt, vect_remove_stores) (supportable_widening_operation): Likewise. From-SVN: r263147
2018-07-31[28/46] Use stmt_vec_info instead of gimple stmts internally (part 1)Richard Sandiford1-1/+1
This first part makes functions use stmt_vec_infos instead of gimple stmts in cases where the stmt_vec_info was already available and where the change is mechanical. Most of it is just replacing "stmt" with "stmt_info". 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vect-data-refs.c (vect_slp_analyze_node_dependences): (vect_check_gather_scatter, vect_create_data_ref_ptr, bump_vector_ptr) (vect_permute_store_chain, vect_setup_realignment) (vect_permute_load_chain, vect_shift_permute_load_chain) (vect_transform_grouped_load): Use stmt_vec_info rather than gimple stmts internally, and when passing values to other vectorizer routines. * tree-vect-loop-manip.c (vect_can_advance_ivs_p): Likewise. * tree-vect-loop.c (vect_analyze_scalar_cycles_1) (vect_analyze_loop_operations, get_initial_def_for_reduction) (vect_create_epilog_for_reduction, vectorize_fold_left_reduction) (vectorizable_reduction, vectorizable_induction) (vectorizable_live_operation, vect_transform_loop_stmt) (vect_transform_loop): Likewise. * tree-vect-patterns.c (vect_reassociating_reduction_p) (vect_recog_widen_op_pattern, vect_recog_mixed_size_cond_pattern) (vect_recog_bool_pattern, vect_recog_gather_scatter_pattern): Likewise. * tree-vect-slp.c (vect_analyze_slp_instance): Likewise. (vect_slp_analyze_node_operations_1): Likewise. * tree-vect-stmts.c (vect_mark_relevant, process_use) (exist_non_indexing_operands_for_use_p, vect_init_vector_1) (vect_mark_stmts_to_be_vectorized, vect_get_vec_def_for_operand) (vect_finish_stmt_generation_1, get_group_load_store_type) (get_load_store_type, vect_build_gather_load_calls) (vectorizable_bswap, vectorizable_call, vectorizable_simd_clone_call) (vect_create_vectorized_demotion_stmts, vectorizable_conversion) (vectorizable_assignment, vectorizable_shift, vectorizable_operation) (vectorizable_store, vectorizable_load, vectorizable_condition) (vectorizable_comparison, vect_analyze_stmt, vect_transform_stmt) (supportable_widening_operation): Likewise. (vect_get_vector_types_for_stmt): Likewise. * tree-vectorizer.h (vect_dr_behavior): Likewise. From-SVN: r263143
2018-07-31[25/46] Make get_earlier/later_stmt take and return stmt_vec_infosRichard Sandiford1-53/+21
...and also make vect_find_last_scalar_stmt_in_slp return a stmt_vec_info. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (get_earlier_stmt, get_later_stmt): Take and return stmt_vec_infos rather than gimple stmts. Do not accept null arguments. (vect_find_last_scalar_stmt_in_slp): Return a stmt_vec_info instead of a gimple stmt. * tree-vect-slp.c (vect_find_last_scalar_stmt_in_slp): Likewise. Update use of get_later_stmt. (vect_get_constant_vectors): Update call accordingly. (vect_schedule_slp_instance): Likewise * tree-vect-data-refs.c (vect_slp_analyze_node_dependences): Likewise. (vect_slp_analyze_instance_dependence): Likewise. (vect_preserves_scalar_order_p): Update use of get_earlier_stmt. From-SVN: r263140
2018-07-31[24/46] Make stmt_info_for_cost use a stmt_vec_infoRichard Sandiford1-5/+2
This patch makes stmt_info_for_cost carry a stmt_vec_info instead of a gimple stmt. The structure is internal to the vectoriser, so targets aren't affected. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (stmt_info_for_cost::stmt): Replace with... (stmt_info_for_cost::stmt_info): ...this new field. (add_stmt_costs): Update accordingly. * tree-vect-loop.c (vect_compute_single_scalar_iteration_cost) (vect_get_known_peeling_cost): Likewise. (vect_estimate_min_profitable_iters): Likewise. * tree-vect-stmts.c (record_stmt_cost): Likewise. From-SVN: r263139
2018-07-31[23/46] Make LOOP_VINFO_MAY_MISALIGN_STMTS use stmt_vec_infoRichard Sandiford1-1/+1
This patch changes LOOP_VINFO_MAY_MISALIGN_STMTS from an auto_vec<gimple *> to an auto_vec<stmt_vec_info>. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (_loop_vec_info::may_misalign_stmts): Change from an auto_vec<gimple *> to an auto_vec<stmt_vec_info>. * tree-vect-data-refs.c (vect_enhance_data_refs_alignment): Update accordingly. * tree-vect-loop-manip.c (vect_create_cond_for_align_checks): Likewise. From-SVN: r263138
2018-07-31[22/46] Make DR_GROUP_SAME_DR_STMT a stmt_vec_infoRichard Sandiford1-1/+1
This patch changes STMT_VINFO_SAME_DR_STMT from a gimple stmt to a stmt_vec_info. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (_stmt_vec_info::same_dr_stmt): Change from a gimple stmt to a stmt_vec_info. * tree-vect-stmts.c (vectorizable_load): Update accordingly. From-SVN: r263137
2018-07-31[21/46] Make grouped_stores and reduction_chains use stmt_vec_infosRichard Sandiford1-2/+2
This patch changes the SLP lists grouped_stores and reduction_chains from auto_vec<gimple *> to auto_vec<stmt_vec_info>. It was easier to do them together due to the way vect_analyze_slp is structured. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (vec_info::grouped_stores): Change from an auto_vec<gimple *> to an auto_vec<stmt_vec_info>. (_loop_vec_info::reduction_chains): Likewise. * tree-vect-loop.c (vect_fixup_scalar_cycles_with_patterns): Update accordingly. * tree-vect-slp.c (vect_analyze_slp): Likewise. From-SVN: r263136
2018-07-31[20/46] Make *FIRST_ELEMENT and *NEXT_ELEMENT stmt_vec_infosRichard Sandiford1-2/+2
This patch changes {REDUC,DR}_GROUP_{FIRST,NEXT} element from a gimple stmt to stmt_vec_info. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (_stmt_vec_info::first_element): Change from a gimple stmt to a stmt_vec_info. (_stmt_vec_info::next_element): Likewise. * tree-vect-data-refs.c (vect_update_misalignment_for_peel) (vect_slp_analyze_and_verify_node_alignment) (vect_analyze_group_access_1, vect_analyze_group_access) (vect_small_gap_p, vect_prune_runtime_alias_test_list) (vect_create_data_ref_ptr, vect_record_grouped_load_vectors) (vect_supportable_dr_alignment): Update accordingly. * tree-vect-loop.c (vect_fixup_reduc_chain): Likewise. (vect_fixup_scalar_cycles_with_patterns, vect_is_slp_reduction) (vect_is_simple_reduction, vectorizable_reduction): Likewise. * tree-vect-patterns.c (vect_reassociating_reduction_p): Likewise. * tree-vect-slp.c (vect_build_slp_tree_1) (vect_attempt_slp_rearrange_stmts, vect_supported_load_permutation_p) (vect_split_slp_store_group, vect_analyze_slp_instance) (vect_analyze_slp, vect_transform_slp_perm_load): Likewise. * tree-vect-stmts.c (vect_model_store_cost, vect_model_load_cost) (get_group_load_store_type, get_load_store_type) (get_group_alias_ptr_type, vectorizable_store, vectorizable_load) (vect_transform_stmt, vect_remove_stores): Likewise. From-SVN: r263135
2018-07-31[19/46] Make vect_dr_stmt return a stmt_vec_infoRichard Sandiford1-2/+2
This patch makes vect_dr_stmt return a stmt_vec_info instead of a gimple stmt. Rather than retain a separate gimple stmt variable in cases where both existed, the patch replaces uses of the gimple variable with the uses of the stmt_vec_info. Later patches do this more generally. Many things that are keyed off a data_reference would these days be better keyed off a stmt_vec_info, but it's more convenient to do that later in the series. The vect_dr_size calls that are left over do still benefit from this patch. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (vect_dr_stmt): Return a stmt_vec_info rather than a gimple stmt. * tree-vect-data-refs.c (vect_analyze_data_ref_dependence) (vect_slp_analyze_data_ref_dependence, vect_record_base_alignments) (vect_calculate_target_alignmentm, vect_compute_data_ref_alignment) (vect_update_misalignment_for_peel, vect_verify_datarefs_alignment) (vector_alignment_reachable_p, vect_get_data_access_cost) (vect_get_peeling_costs_all_drs, vect_peeling_hash_get_lowest_cost) (vect_peeling_supportable, vect_enhance_data_refs_alignment) (vect_find_same_alignment_drs, vect_analyze_data_refs_alignment) (vect_analyze_group_access_1, vect_analyze_group_access) (vect_analyze_data_ref_access, vect_analyze_data_ref_accesses) (vect_vfa_access_size, vect_small_gap_p, vect_analyze_data_refs) (vect_supportable_dr_alignment): Remove vinfo_for_stmt from the result of vect_dr_stmt and use the stmt_vec_info instead of the associated gimple stmt. * tree-vect-loop-manip.c (get_misalign_in_elems): Likewise. (vect_gen_prolog_loop_niters): Likewise. * tree-vect-loop.c (vect_analyze_loop_2): Likewise. From-SVN: r263134
2018-07-31[18/46] Make SLP_TREE_SCALAR_STMTS a vec<stmt_vec_info>Richard Sandiford1-1/+1
This patch changes SLP_TREE_SCALAR_STMTS from a vec<gimple *> to a vec<stmt_vec_info>. It's longer than the previous conversions but mostly mechanical. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (_slp_tree::stmts): Change from a vec<gimple *> to a vec<stmt_vec_info>. * tree-vect-slp.c (vect_free_slp_tree): Update accordingly. (vect_create_new_slp_node): Take a vec<gimple *> instead of a vec<stmt_vec_info>. (_slp_oprnd_info::def_stmts): Change from a vec<gimple *> to a vec<stmt_vec_info>. (bst_traits::value_type, bst_traits::value_type): Likewise. (bst_traits::hash): Update accordingly. (vect_get_and_check_slp_defs): Change the stmts parameter from a vec<gimple *> to a vec<stmt_vec_info>. (vect_two_operations_perm_ok_p, vect_build_slp_tree_1): Likewise. (vect_build_slp_tree): Likewise. (vect_build_slp_tree_2): Likewise. Update uses of SLP_TREE_SCALAR_STMTS. (vect_print_slp_tree): Update uses of SLP_TREE_SCALAR_STMTS. (vect_mark_slp_stmts, vect_mark_slp_stmts_relevant) (vect_slp_rearrange_stmts, vect_attempt_slp_rearrange_stmts) (vect_supported_load_permutation_p, vect_find_last_scalar_stmt_in_slp) (vect_detect_hybrid_slp_stmts, vect_slp_analyze_node_operations_1) (vect_slp_analyze_node_operations, vect_slp_analyze_operations) (vect_bb_slp_scalar_cost, vect_slp_analyze_bb_1) (vect_get_constant_vectors, vect_get_slp_defs) (vect_transform_slp_perm_load, vect_schedule_slp_instance) (vect_remove_slp_scalar_calls, vect_schedule_slp): Likewise. (vect_analyze_slp_instance): Build up a vec of stmt_vec_infos instead of gimple stmts. * tree-vect-data-refs.c (vect_slp_analyze_node_dependences): Change the stores parameter for a vec<gimple *> to a vec<stmt_vec_info>. (vect_slp_analyze_instance_dependence): Update uses of SLP_TREE_SCALAR_STMTS. (vect_slp_analyze_and_verify_node_alignment): Likewise. (vect_slp_analyze_and_verify_instance_alignment): Likewise. * tree-vect-loop.c (neutral_op_for_slp_reduction): Likewise. (get_initial_defs_for_reduction): Likewise. (vect_create_epilog_for_reduction): Likewise. (vectorize_fold_left_reduction): Likewise. * tree-vect-stmts.c (vect_prologue_cost_for_slp_op): Likewise. (vect_model_simple_cost, vectorizable_shift, vectorizable_load) (can_vectorize_live_stmts): Likewise. From-SVN: r263133
2018-07-31[17/46] Make LOOP_VINFO_REDUCTIONS an auto_vec<stmt_vec_info>Richard Sandiford1-3/+3
This patch changes LOOP_VINFO_REDUCTIONS from an auto_vec<gimple *> to an auto_vec<stmt_vec_info>. It also changes the associated vect_force_simple_reduction so that it takes and returns stmt_vec_infos instead of gimple stmts. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (_loop_vec_info::reductions): Change from an auto_vec<gimple *> to an auto_vec<stmt_vec_info>. (vect_force_simple_reduction): Take and return stmt_vec_infos rather than gimple stmts. * tree-parloops.c (valid_reduction_p): Take a stmt_vec_info instead of a gimple stmt. (gather_scalar_reductions): Update after above interface changes. * tree-vect-loop.c (vect_analyze_scalar_cycles_1): Likewise. (vect_is_simple_reduction): Take and return stmt_vec_infos rather than gimple stmts. (vect_force_simple_reduction): Likewise. * tree-vect-patterns.c (vect_pattern_recog_1): Update use of LOOP_VINFO_REDUCTIONS. * tree-vect-slp.c (vect_analyze_slp_instance): Likewise. From-SVN: r263132
2018-07-31[16/46] Make STMT_VINFO_REDUC_DEF a stmt_vec_infoRichard Sandiford1-1/+1
This patch changes STMT_VINFO_REDUC_DEF from a gimple stmt to a stmt_vec_info. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (_stmt_vec_info::reduc_def): Change from a gimple stmt to a stmt_vec_info. * tree-vect-loop.c (vect_active_double_reduction_p) (vect_force_simple_reduction, vectorizable_reduction): Update accordingly. From-SVN: r263131
2018-07-31[15/46] Make SLP_TREE_VEC_STMTS a vec<stmt_vec_info>Richard Sandiford1-1/+1
This patch changes SLP_TREE_VEC_STMTS from a vec<gimple *> to a vec<stmt_vec_info>. This involved making the same change to the phis vector in vectorizable_reduction, since SLP_TREE_VEC_STMTS is spliced into it here: phis.splice (SLP_TREE_VEC_STMTS (slp_node_instance->reduc_phis)); 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (_slp_tree::vec_stmts): Change from a vec<gimple *> to a vec<stmt_vec_info>. * tree-vect-loop.c (vect_create_epilog_for_reduction): Change the reduction_phis argument from a vec<gimple *> to a vec<stmt_vec_info>. (vectorizable_reduction): Likewise the phis local variable that is passed to vect_create_epilog_for_reduction. Update for new type of SLP_TREE_VEC_STMTS. (vectorizable_induction): Update for new type of SLP_TREE_VEC_STMTS. (vectorizable_live_operation): Likewise. * tree-vect-slp.c (vect_get_slp_vect_defs): Likewise. (vect_transform_slp_perm_load, vect_schedule_slp_instance): Likewise. From-SVN: r263130
2018-07-31[14/46] Make STMT_VINFO_VEC_STMT a stmt_vec_infoRichard Sandiford1-5/+5
This patch changes STMT_VINFO_VEC_STMT from a gimple stmt to a stmt_vec_info and makes the vectorizable_* routines pass back a stmt_vec_info to vect_transform_stmt. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (_stmt_vec_info::vectorized_stmt): Change from a gimple stmt to a stmt_vec_info. (vectorizable_condition, vectorizable_live_operation) (vectorizable_reduction, vectorizable_induction): Pass back the vectorized statement as a stmt_vec_info. * tree-vect-data-refs.c (vect_record_grouped_load_vectors): Update use of STMT_VINFO_VEC_STMT. * tree-vect-loop.c (vect_create_epilog_for_reduction): Likewise, accumulating the inner phis that feed the STMT_VINFO_VEC_STMT as stmt_vec_infos rather than gimple stmts. (vectorize_fold_left_reduction): Change vec_stmt from a gimple stmt to a stmt_vec_info. (vectorizable_live_operation): Likewise. (vectorizable_reduction, vectorizable_induction): Likewise, updating use of STMT_VINFO_VEC_STMT. * tree-vect-stmts.c (vect_get_vec_def_for_operand_1): Update use of STMT_VINFO_VEC_STMT. (vect_build_gather_load_calls, vectorizable_bswap, vectorizable_call) (vectorizable_simd_clone_call, vectorizable_conversion) (vectorizable_assignment, vectorizable_shift, vectorizable_operation) (vectorizable_store, vectorizable_load, vectorizable_condition) (vectorizable_comparison, can_vectorize_live_stmts): Change vec_stmt from a gimple stmt to a stmt_vec_info. (vect_transform_stmt): Update use of STMT_VINFO_VEC_STMT. Pass a pointer to a stmt_vec_info to the vectorizable_* routines. From-SVN: r263129
2018-07-31[13/46] Make STMT_VINFO_RELATED_STMT a stmt_vec_infoRichard Sandiford1-11/+3
This patch changes STMT_VINFO_RELATED_STMT from a gimple stmt to a stmt_vec_info. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (_stmt_vec_info::related_stmt): Change from a gimple stmt to a stmt_vec_info. (is_pattern_stmt_p): Update accordingly. * tree-vect-data-refs.c (vect_preserves_scalar_order_p): Likewise. (vect_record_grouped_load_vectors): Likewise. * tree-vect-loop.c (vect_determine_vf_for_stmt): Likewise. (vect_fixup_reduc_chain, vect_update_vf_for_slp): Likewise. (vect_model_reduction_cost): Likewise. (vect_create_epilog_for_reduction): Likewise. (vectorizable_reduction, vectorizable_induction): Likewise. * tree-vect-patterns.c (vect_init_pattern_stmt): Likewise. Return the stmt_vec_info for the pattern statement. (vect_set_pattern_stmt): Update use of STMT_VINFO_RELATED_STMT. (vect_split_statement, vect_mark_pattern_stmts): Likewise. * tree-vect-slp.c (vect_detect_hybrid_slp_stmts): Likewise. (vect_detect_hybrid_slp, vect_get_slp_defs): Likewise. * tree-vect-stmts.c (vect_mark_relevant): Likewise. (vect_get_vec_def_for_operand_1, vectorizable_call): Likewise. (vectorizable_simd_clone_call, vect_analyze_stmt, new_stmt_vec_info) (free_stmt_vec_info, vect_is_simple_use): Likewise. From-SVN: r263128
2018-07-31[12/46] Make vect_finish_stmt_generation return a stmt_vec_infoRichard Sandiford1-3/+3
This patch makes vect_finish_replace_stmt and vect_finish_stmt_generation return the stmt_vec_info for the vectorised statement, so that the caller doesn't need a separate vinfo_for_stmt to get at it. This involved changing the structure of the statement-generating loops so that they use narrow scopes for the vectorised gimple statements and use the existing (wider) scopes for the associated stmt_vec_infos. This helps with gimple stmt->stmt_vec_info changes further down the line. The way we do this generation is another area ripe for clean-up, but that's too much of a rabbit-hole for this series. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (vect_finish_replace_stmt): Return a stmt_vec_info (vect_finish_stmt_generation): Likewise. * tree-vect-stmts.c (vect_finish_stmt_generation_1): Likewise. (vect_finish_replace_stmt, vect_finish_stmt_generation): Likewise. (vect_build_gather_load_calls): Use the return value of the above functions instead of a separate call to vinfo_for_stmt. Use narrow scopes for the input gimple stmt and wider scopes for the associated stmt_vec_info. Use vec_info::lookup_def when setting these stmt_vec_infos from an SSA_NAME definition. (vectorizable_bswap, vectorizable_call, vectorizable_simd_clone_call) (vect_create_vectorized_demotion_stmts, vectorizable_conversion) (vectorizable_assignment, vectorizable_shift, vectorizable_operation) (vectorizable_store, vectorizable_load, vectorizable_condition) (vectorizable_comparison): Likewise. * tree-vect-loop.c (vectorize_fold_left_reduction): Likewise. (vectorizable_reduction): Likewise. From-SVN: r263127
2018-07-31[11/46] Pass back a stmt_vec_info from vect_is_simple_useRichard Sandiford1-2/+3
This patch makes vect_is_simple_use pass back a stmt_vec_info to those callers that want it. Most users only need the stmt_vec_info but some need the gimple stmt too. It's probably high time we added a class to represent "simple operands" instead, but I have a separate series that tries to clean up how operands are handled (with a view to allowing mixed vector sizes). 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (vect_is_simple_use): Add an optional stmt_vec_info * parameter before the optional gimple **. * tree-vect-stmts.c (vect_is_simple_use): Likewise. (process_use, vect_get_vec_def_for_operand_1): Update callers. (vect_get_vec_def_for_operand, vectorizable_shift): Likewise. * tree-vect-loop.c (vectorizable_reduction): Likewise. (vectorizable_live_operation): Likewise. * tree-vect-patterns.c (type_conversion_p): Likewise. (vect_look_through_possible_promotion): Likewise. (vect_recog_rotate_pattern): Likewise. * tree-vect-slp.c (vect_get_and_check_slp_defs): Likewise. From-SVN: r263126
2018-07-31[10/46] Temporarily make stmt_vec_info a classRichard Sandiford1-5/+37
This patch turns stmt_vec_info into an unspeakably bad wrapper class and adds an implicit conversion to the associated gimple stmt. Having this conversion makes the rest of the series easier to write, but since the class goes away again at the end of the series, I've not bothered adding any comments or tried to make it pretty. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (stmt_vec_info): Temporarily change from a typedef to a wrapper class. (NULL_STMT_VEC_INFO): New macro. (vec_info::stmt_infos): Change to vec<stmt_vec_info>. (stmt_vec_info::operator*): New function. (stmt_vec_info::operator gimple *): Likewise. (set_vinfo_for_stmt): Use NULL_STMT_VEC_INFO. (add_stmt_costs): Likewise. * tree-vect-loop-manip.c (iv_phi_p): Likewise. * tree-vect-loop.c (vect_compute_single_scalar_iteration_cost) (vect_get_known_peeling_cost): Likewise. (vect_estimate_min_profitable_iters): Likewise. * tree-vect-patterns.c (vect_init_pattern_stmt): Likewise. * tree-vect-slp.c (vect_remove_slp_scalar_calls): Likewise. * tree-vect-stmts.c (vect_build_gather_load_calls): Likewise. (vectorizable_store, free_stmt_vec_infos): Likewise. (new_stmt_vec_info): Change return type of xcalloc to _stmt_vec_info *. From-SVN: r263125
2018-07-31[09/46] Add vec_info::lookup_single_useRichard Sandiford1-0/+1
This patch adds a helper function for seeing whether there is a single user of an SSA name, and whether that user has a stmt_vec_info. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (vec_info::lookup_single_use): Declare. * tree-vectorizer.c (vec_info::lookup_single_use): New function. * tree-vect-loop.c (vectorizable_reduction): Use it instead of a single_imm_use-based sequence. * tree-vect-stmts.c (supportable_widening_operation): Likewise. From-SVN: r263124
2018-07-31[08/46] Add vec_info::lookup_defRichard Sandiford1-0/+1
This patch adds a vec_info helper for checking whether an operand is an SSA_NAME that is defined in the vectorisable region. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (vec_info::lookup_def): Declare. * tree-vectorizer.c (vec_info::lookup_def): New function. * tree-vect-patterns.c (vect_get_internal_def): Use it. (vect_widened_op_tree): Likewise. * tree-vect-stmts.c (vect_is_simple_use): Likewise. * tree-vect-loop.c (vect_analyze_loop_operations): Likewise. (vectorizable_reduction): Likewise. (vect_valid_reduction_input_p): Take a stmt_vec_info instead of a gimple *. (vect_is_slp_reduction): Update calls accordingly. Use vec_info::lookup_def. (vect_is_simple_reduction): Likewise * tree-vect-slp.c (vect_detect_hybrid_slp_1): Use vec_info::lookup_def. From-SVN: r263123
2018-07-31[07/46] Add vec_info::lookup_stmtRichard Sandiford1-0/+1
This patch adds a vec_info replacement for vinfo_for_stmt. The main difference is that the new routine can cope with arbitrary statements, so there's no need to call vect_stmt_in_region_p first. The patch only converts calls that are still needed at the end of the series. Later patches get rid of most other calls to vinfo_for_stmt. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (vec_info::lookup_stmt): Declare. * tree-vectorizer.c (vec_info::lookup_stmt): New function. * tree-vect-loop.c (vect_determine_vf_for_stmt): Use it instead of vinfo_for_stmt. (vect_determine_vectorization_factor, vect_analyze_scalar_cycles_1) (vect_compute_single_scalar_iteration_cost, vect_analyze_loop_form) (vect_update_vf_for_slp, vect_analyze_loop_operations) (vect_is_slp_reduction, vectorizable_induction) (vect_transform_loop_stmt, vect_transform_loop): Likewise. * tree-vect-patterns.c (vect_init_pattern_stmt): (vect_determine_min_output_precision_1, vect_determine_precisions) (vect_pattern_recog): Likewise. * tree-vect-stmts.c (vect_analyze_stmt, vect_transform_stmt): Likewise. * config/powerpcspe/powerpcspe.c (rs6000_density_test): Likewise. * config/rs6000/rs6000.c (rs6000_density_test): Likewise. * tree-vect-slp.c (vect_detect_hybrid_slp_stmts): Likewise. (vect_detect_hybrid_slp_1, vect_detect_hybrid_slp_2) (vect_detect_hybrid_slp): Likewise. Change the walk_stmt_info info field from a loop to a loop_vec_info. From-SVN: r263122
2018-07-31[06/46] Add vec_info::add_stmtRichard Sandiford1-2/+6
This patch adds a vec_info function for allocating and setting stmt_vec_infos. It's the start of a long process of removing the global stmt_vec_info array. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (stmt_vec_info): Move typedef earlier in file. (vec_info::add_stmt): Declare. * tree-vectorizer.c (vec_info::add_stmt): New function. * tree-vect-data-refs.c (vect_create_data_ref_ptr): Use it. * tree-vect-loop.c (_loop_vec_info::_loop_vec_info): Likewise. (vect_create_epilog_for_reduction, vectorizable_reduction): Likewise. (vectorizable_induction): Likewise. * tree-vect-slp.c (_bb_vec_info::_bb_vec_info): Likewise. * tree-vect-stmts.c (vect_finish_stmt_generation_1): Likewise. (vectorizable_simd_clone_call, vectorizable_store): Likewise. (vectorizable_load): Likewise. * tree-vect-patterns.c (vect_init_pattern_stmt): Likewise. (vect_recog_bool_pattern, vect_recog_mask_conversion_pattern) (vect_recog_gather_scatter_pattern): Likewise. (append_pattern_def_seq): Likewise. Remove a check that is performed by add_stmt itself. From-SVN: r263121
2018-07-31[03/46] Remove unnecessary update of NUM_SLP_USESRichard Sandiford1-1/+1
vect_free_slp_tree had: gimple *stmt; FOR_EACH_VEC_ELT (SLP_TREE_SCALAR_STMTS (node), i, stmt) /* After transform some stmts are removed and thus their vinfo is gone. */ if (vinfo_for_stmt (stmt)) { gcc_assert (STMT_VINFO_NUM_SLP_USES (vinfo_for_stmt (stmt)) > 0); STMT_VINFO_NUM_SLP_USES (vinfo_for_stmt (stmt))--; } But after transform this update is redundant even for statements that do exist, so it seems better to skip this loop for the final teardown. 2018-07-31 Richard Sandiford <richard.sandiford@arm.com> gcc/ * tree-vectorizer.h (vect_free_slp_instance): Add a final_p parameter. * tree-vect-slp.c (vect_free_slp_tree): Likewise. Don't update STMT_VINFO_NUM_SLP_USES when it's true. (vect_free_slp_instance): Add a final_p parameter and pass it to vect_free_slp_tree. (vect_build_slp_tree_2): Update call to vect_free_slp_instance. (vect_analyze_slp_instance): Likewise. (vect_slp_analyze_operations): Likewise. (vect_slp_analyze_bb_1): Likewise. * tree-vectorizer.c (vec_info): Likewise. * tree-vect-loop.c (vect_transform_loop): Likewise. From-SVN: r263118
2018-07-03[14/n] PR85694: Rework overwidening detectionRichard Sandiford1-0/+15
This patch is the main part of PR85694. The aim is to recognise at least: signed char *a, *b, *c; ... for (int i = 0; i < 2048; i++) c[i] = (a[i] + b[i]) >> 1; as an over-widening pattern, since the addition and shift can be done on shorts rather than ints. However, it ended up being a lot more general than that. The current over-widening pattern detection is limited to a few simple cases: logical ops with immediate second operands, and shifts by a constant. These cases are enough for common pixel-format conversion and can be detected in a peephole way. The loop above requires two generalisations of the current code: support for addition as well as logical ops, and support for non-constant second operands. These are harder to detect in the same peephole way, so the patch tries to take a more global approach. The idea is to get information about the minimum operation width in two ways: (1) by using the range information attached to the SSA_NAMEs (effectively a forward walk, since the range info is context-independent). (2) by back-propagating the number of output bits required by users of the result. As explained in the comments, there's a balance to be struck between narrowing an individual operation and fitting in with the surrounding code. The approach is pretty conservative: if we could narrow an operation to N bits without changing its semantics, it's OK to do that if: - no operations later in the chain require more than N bits; or - all internally-defined inputs are extended from N bits or fewer, and at least one of them is single-use. See the comments for the rationale. I didn't bother adding STMT_VINFO_* wrappers for the new fields since the code seemed more readable without. 2018-06-20 Richard Sandiford <richard.sandiford@arm.com> gcc/ * poly-int.h (print_hex): New function. * dumpfile.h (dump_dec, dump_hex): Declare. * dumpfile.c (dump_dec, dump_hex): New poly_wide_int functions. * tree-vectorizer.h (_stmt_vec_info): Add min_output_precision, min_input_precision, operation_precision and operation_sign. * tree-vect-patterns.c (vect_get_range_info): New function. (vect_same_loop_or_bb_p, vect_single_imm_use) (vect_operation_fits_smaller_type): Delete. (vect_look_through_possible_promotion): Add an optional single_use_p parameter. (vect_recog_over_widening_pattern): Rewrite to use new stmt_vec_info infomration. Handle one operation at a time. (vect_recog_cast_forwprop_pattern, vect_narrowable_type_p) (vect_truncatable_operation_p, vect_set_operation_type) (vect_set_min_input_precision): New functions. (vect_determine_min_output_precision_1): Likewise. (vect_determine_min_output_precision): Likewise. (vect_determine_precisions_from_range): Likewise. (vect_determine_precisions_from_users): Likewise. (vect_determine_stmt_precisions, vect_determine_precisions): Likewise. (vect_vect_recog_func_ptrs): Put over_widening first. Add cast_forwprop. (vect_pattern_recog): Call vect_determine_precisions. gcc/testsuite/ * gcc.dg/vect/vect-widen-mult-u8-u32.c: Check specifically for a widen_mult pattern. * gcc.dg/vect/vect-over-widen-1.c: Update the scan tests for new over-widening messages. * gcc.dg/vect/vect-over-widen-1-big-array.c: Likewise. * gcc.dg/vect/vect-over-widen-2.c: Likewise. * gcc.dg/vect/vect-over-widen-2-big-array.c: Likewise. * gcc.dg/vect/vect-over-widen-3.c: Likewise. * gcc.dg/vect/vect-over-widen-3-big-array.c: Likewise. * gcc.dg/vect/vect-over-widen-4.c: Likewise. * gcc.dg/vect/vect-over-widen-4-big-array.c: Likewise. * gcc.dg/vect/bb-slp-over-widen-1.c: New test. * gcc.dg/vect/bb-slp-over-widen-2.c: Likewise. * gcc.dg/vect/vect-over-widen-5.c: Likewise. * gcc.dg/vect/vect-over-widen-6.c: Likewise. * gcc.dg/vect/vect-over-widen-7.c: Likewise. * gcc.dg/vect/vect-over-widen-8.c: Likewise. * gcc.dg/vect/vect-over-widen-9.c: Likewise. * gcc.dg/vect/vect-over-widen-10.c: Likewise. * gcc.dg/vect/vect-over-widen-11.c: Likewise. * gcc.dg/vect/vect-over-widen-12.c: Likewise. * gcc.dg/vect/vect-over-widen-13.c: Likewise. * gcc.dg/vect/vect-over-widen-14.c: Likewise. * gcc.dg/vect/vect-over-widen-15.c: Likewise. * gcc.dg/vect/vect-over-widen-16.c: Likewise. * gcc.dg/vect/vect-over-widen-17.c: Likewise. * gcc.dg/vect/vect-over-widen-18.c: Likewise. * gcc.dg/vect/vect-over-widen-19.c: Likewise. * gcc.dg/vect/vect-over-widen-20.c: Likewise. * gcc.dg/vect/vect-over-widen-21.c: Likewise. From-SVN: r262333