| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
From-SVN: r267494
|
|
As Jakub pointed out, if we narrow a plus, minus or mult operation based
on the number of bits that consumers need, we have to convert a signed
operation to an unsigned one in order to avoid new undefined behaviour.
This patch does that and generalises vect_convert_input and
vect_recog_over_widening_pattern to cope with the extra casts.
(The changes to both functions are covered by existing tests.)
2018-12-03 Richard Sandiford <richard.sandiford@arm.com>
gcc/
PR tree-optimization/88064
* tree-vect-patterns.c (vect_convert_input): Convert the result of
an existing cast if it has the right width but the wrong sign.
Do not test the signedness of the required result when
considering whether to split an existing cast; instead split to
a type with the same signedness as the source of the cast, then
convert it to the opposite signedness where necessary.
(vect_recog_over_widening_pattern): Handle sign changes between
the final PLUS_EXPR and the RSHIFT_EXPR.
(vect_recog_average_pattern): Use an unsigned operation when
truncating an addition, subtraction or multiplication. Cast the
result back to the "real" signedness before promoting.
gcc/testsuite/
PR tree-optimization/88064
* gcc.dg/vect/vect-over-widen-23.c: New test.
From-SVN: r266829
|
|
tree-vect-loop.c:4689)
2018-11-29 Richard Biener <rguenther@suse.de>
PR tree-optimization/88243
* tree-vect-patterns.c (vect_mark_pattern_stmts): Set the def
type of all pattern-sequence stmts to vect_internal_def.
* gcc.dg/torture/pr88243.c: New testcase.
From-SVN: r266615
|
|
PR tree-optimization/87546
* tree-vect-patterns.c (vect_look_through_possible_promotion): Add
min_precision variable, initially set it to orig_precision, only does
something if op_type's precision is <= min_precision and update
min_precision whenever calling set_op.
* gcc.dg/vect/O3-pr87546.c: New test.
From-SVN: r266237
|
|
This long patch only does one simple thing, adds an explicit function
parameter to predicates stmt_could_throw_p, stmt_can_throw_external
and stmt_can_throw_internal.
My motivation was ability to use stmt_can_throw_external in IPA
analysis phase without the need to push cfun. As I have discovered,
we were already doing that in cgraph.c, which this patch avoids as
well. In the process, I had to add a struct function parameter to
stmt_could_throw_p and decided to also change the interface of
stmt_can_throw_internal just for the sake of some minimal consistency.
In the process I have discovered that calling method
cgraph_node::create_version_clone_with_body (used by ipa-split,
ipa-sra, OMP simd and multiple_target) leads to calls of
stmt_can_throw_external with NULL cfun. I have worked around this by
making stmt_can_throw_external and stmt_could_throw_p gracefully
accept NULL and just be pessimistic in that case. The problem with
fixing this in a better way is that struct function for the clone is
created after cloning edges where we attempt to push the yet not
existing cfun, and moving it before would require a bit of surgery in
tree-inline.c. A slightly hackish but simpler fix might be to
explicitely pass the "old" function to symbol_table::create_edge
because it should be just as good at that moment. In any event, that
is a topic for another patch.
I believe that currently we incorrectly use cfun in
maybe_clean_eh_stmt_fn and maybe_duplicate_eh_stmt_fn, both in
tree-eh.c, and so I have fixed these cases too. The bulk of other
changes is just mechanical adding of cfun to all users.
Bootstrapped and tested on x86_64-linux (also with extra NULLing and
restoring cfun to double check it is not used in a place I missed), OK
for trunk?
Thanks,
Martin
2018-10-22 Martin Jambor <mjambor@suse.cz>
* tree-eh.h (stmt_could_throw_p): Add function parameter.
(stmt_can_throw_external): Likewise.
(stmt_can_throw_internal): Likewise.
* tree-eh.c (lower_eh_constructs_2): Pass cfun to stmt_could_throw_p.
(lower_eh_constructs_2): Likewise.
(stmt_could_throw_p): Add fun parameter, use it instead of cfun.
(stmt_can_throw_external): Likewise.
(stmt_can_throw_internal): Likewise.
(maybe_clean_eh_stmt_fn): Pass cfun to stmt_could_throw_p.
(maybe_clean_or_replace_eh_stmt): Pass cfun to stmt_could_throw_p.
(maybe_duplicate_eh_stmt_fn): Pass new_fun to stmt_could_throw_p.
(maybe_duplicate_eh_stmt): Pass cfun to stmt_could_throw_p.
(pass_lower_eh_dispatch::execute): Pass cfun to
stmt_can_throw_external.
(cleanup_empty_eh): Likewise.
(verify_eh_edges): Pass cfun to stmt_could_throw_p.
* cgraph.c (cgraph_edge::set_call_stmt): Pass a function to
stmt_can_throw_external instead of pushing it to cfun.
(symbol_table::create_edge): Likewise.
* gimple-fold.c (fold_builtin_atomic_compare_exchange): Pass cfun to
stmt_can_throw_internal.
* gimple-ssa-evrp.c (evrp_dom_walker::before_dom_children): Pass cfun
to stmt_could_throw_p.
* gimple-ssa-store-merging.c (handled_load): Pass cfun to
stmt_can_throw_internal.
(pass_store_merging::execute): Likewise.
* gimple-ssa-strength-reduction.c
(find_candidates_dom_walker::before_dom_children): Pass cfun to
stmt_could_throw_p.
* gimplify-me.c (gimple_regimplify_operands): Pass cfun to
stmt_can_throw_internal.
* ipa-pure-const.c (check_call): Pass cfun to stmt_could_throw_p and
to stmt_can_throw_external.
(check_stmt): Pass cfun to stmt_could_throw_p.
(check_stmt): Pass cfun to stmt_can_throw_external.
(pass_nothrow::execute): Likewise.
* trans-mem.c (expand_call_tm): Pass cfun to stmt_can_throw_internal.
* tree-cfg.c (is_ctrl_altering_stmt): Pass cfun to
stmt_can_throw_internal.
(verify_gimple_in_cfg): Pass cfun to stmt_could_throw_p.
(stmt_can_terminate_bb_p): Pass cfun to stmt_can_throw_external.
(gimple_purge_dead_eh_edges): Pass cfun to stmt_can_throw_internal.
* tree-complex.c (expand_complex_libcall): Pass cfun to
stmt_could_throw_p and to stmt_can_throw_internal.
(expand_complex_multiplication): Pass cfun to stmt_can_throw_internal.
* tree-inline.c (copy_edges_for_bb): Likewise.
(maybe_move_debug_stmts_to_successors): Likewise.
* tree-outof-ssa.c (ssa_is_replaceable_p): Pass cfun to
stmt_could_throw_p.
* tree-parloops.c (oacc_entry_exit_ok_1): Likewise.
* tree-sra.c (scan_function): Pass cfun to stmt_can_throw_external.
* tree-ssa-alias.c (stmt_kills_ref_p): Pass cfun to
stmt_can_throw_internal.
* tree-ssa-ccp.c (optimize_atomic_bit_test_and): Likewise.
* tree-ssa-dce.c (mark_stmt_if_obviously_necessary): Pass cfun to
stmt_could_throw_p.
(mark_aliased_reaching_defs_necessary_1): Pass cfun to
stmt_can_throw_internal.
* tree-ssa-forwprop.c (pass_forwprop::execute): Likewise.
* tree-ssa-loop-im.c (movement_possibility): Pass cfun to
stmt_could_throw_p.
* tree-ssa-loop-ivopts.c (find_givs_in_stmt_scev): Likewise.
(add_autoinc_candidates): Pass cfun to stmt_can_throw_internal.
* tree-ssa-math-opts.c (pass_cse_reciprocals::execute): Likewise.
(convert_mult_to_fma_1): Likewise.
(convert_to_divmod): Likewise.
* tree-ssa-phiprop.c (propagate_with_phi): Likewise.
* tree-ssa-pre.c (compute_avail): Pass cfun to stmt_could_throw_p.
* tree-ssa-propagate.c
(substitute_and_fold_dom_walker::before_dom_children): Likewise.
* tree-ssa-reassoc.c (suitable_cond_bb): Likewise.
(maybe_optimize_range_tests): Likewise.
(linearize_expr_tree): Likewise.
(reassociate_bb): Likewise.
* tree-ssa-sccvn.c (copy_reference_ops_from_call): Likewise.
* tree-ssa-scopedtables.c (hashable_expr_equal_p): Likewise.
* tree-ssa-strlen.c (adjust_last_stmt): Likewise.
(handle_char_store): Likewise.
* tree-vect-data-refs.c (vect_find_stmt_data_reference): Pass cfun to
stmt_can_throw_internal.
* tree-vect-patterns.c (check_bool_pattern): Pass cfun to
stmt_could_throw_p.
* tree-vect-stmts.c (vect_finish_stmt_generation_1): Likewise.
(vectorizable_call): Pass cfun to stmt_can_throw_internal.
(vectorizable_simd_clone_call): Likewise.
* value-prof.c (gimple_ic): Pass cfun to stmt_could_throw_p.
(gimple_stringop_fixed_value): Likewise.
From-SVN: r265372
|
|
* bitmap.c (bitmap_head::dump): New.
* bitmap.h (bitmap_head): Add dump().
* gimple-ssa-evrp-analyze.c
(evrp_range_analyzer::try_find_new_range): Adjust for value_range API.
(evrp_range_analyzer::set_ssa_range_info): Same.
(evrp_range_analyzer::record_ranges_from_phis): Same.
(evrp_range_analyzer::record_ranges_from_stmt): Same.
* gimple-ssa-evrp.c (evrp_dom_walker::before_dom_children): Same.
* gimple-ssa-sprintf.c (get_int_range): Same.
(format_integer): Same.
(sprintf_dom_walker::handle_gimple_call): Same.
* ipa-cp.c (ipcp_vr_lattice::meet_with_1): Same.
(ipcp_vr_lattice::top_p): Same.
(ipcp_vr_lattice::bottom_p): Same.
(ipcp_vr_lattice::set_to_bottom): Same.
(ipa_vr_operation_and_type_effects): Same.
(propagate_vr_across_jump_function): Same.
(ipcp_store_vr_results): Same.
* ipa-prop.c (struct ipa_vr_ggc_hash_traits): Same.
(ipa_print_node_jump_functions_for_edge): Same.
(ipa_get_value_range): Same.
(ipa_compute_jump_functions_for_edge): Same.
(ipa_write_jump_function): Same.
* tree-ssa-dom.c (simplify_stmt_for_jump_threading): Same.
* tree-ssa-threadedge.c (record_temporary_equivalences_from_phis):
Same.
* vr-values.c (set_value_range_to_nonnegative): Same.
(set_value_range_to_truthvalue): Same.
(vr_values::get_value_range): Same.
(vr_values::set_defs_to_varying): Same.
(vr_values::update_value_range): Same.
(symbolic_range_based_on_p): Same.
(vr_values::op_with_boolean_value_range_p): Same.
(vr_values::extract_range_for_var_from_comparison_expr): Same.
(vr_values::extract_range_from_ssa_name): Same.
(vr_values::extract_range_from_binary_expr): Same.
(vr_values::extract_range_from_unary_expr): Same.
(vr_values::extract_range_from_cond_expr): Same.
(vr_values::extract_range_from_comparison): Same.
(vr_values::check_for_binary_op_overflow): Same.
(vr_values::extract_range_basic): Same.
(vr_values::extract_range_from_assignment): Same.
(compare_ranges): Same.
(compare_range_with_value): Same.
(vr_values::adjust_range_with_scev): Same.
(vrp_valueize): Same.
(vrp_valueize_1): Same.
(vr_values::get_vr_for_comparison): Same.
(vr_values::compare_name_with_value): Same.
(vr_values::compare_names): Same.
(vr_values::vrp_evaluate_conditional): Same.
(find_case_label_ranges): Same.
(vr_values::vrp_visit_switch_stmt): Same.
(vr_values::extract_range_from_phi_node): Same.
(vr_values::simplify_div_or_mod_using_ranges): Same.
(vr_values::simplify_bit_ops_using_ranges): Same.
(test_for_singularity): Same.
(range_fits_type_p): Same.
(vr_values::simplify_cond_using_ranges_1): Same.
(vr_values::simplify_switch_using_ranges): Same.
(vr_values::simplify_float_conversion_using_ranges): Same.
(vr_values::two_valued_val_range_p): Same.
(vr_values::add_equivalence): Move to value_range::equiv_add.
* vr-values.h (vr_values::add_equivalence): Remove.
(VR_INITIALIZER): Remove.
* tree-vrp.c (value_range::set): New.
(value_range::equiv_add): New.
(value_range::value_range): New.
(value_range::deep_copy): New.
(value_range::check): New.
(value_range::equal_p): New.
(value_range::ignore_equivs_equal_p): New.
(value_range::operator==): New.
(value_range::operator!=): New.
(value_range::symbolic_p): New.
(value_range::numeric_p): New.
(value_range::set_undefined): New.
(value_range::set_varying): New.
(value_range::may_contain_p): New.
(value_range::equiv_clear): New.
(value_range::singleton_p): New.
(value_range::intersect): New.
(value_range::dump): New.
(value_range::set_and_canonicalize): New.
(set_value_range): Adjust for value_range API.
(set_value_range_to_undefined): Same.
(set_value_range_to_varying): Same.
(set_and_canonicalize_value_range): Same.
(set_value_range_to_nonnull): Same.
(set_value_range_to_null): Same.
(range_is_null): Same.
(range_is_nonnull): Same.
(range_int_cst_p): Same.
(range_int_cst_singleton_p): Same.
(symbolic_range_p): Same.
(range_includes_zero_p): Same.
(value_range_constant_singleton): Same.
(vrp_set_zero_nonzero_bits): Same.
(ranges_from_anti_range): Same.
(extract_range_into_wide_ints): Same.
(extract_range_from_multiplicative_op): Same.
(set_value_range_with_overflow): Same.
(extract_range_from_binary_expr_1): Same.
(extract_range_from_unary_expr): Same.
(dump_value_range): Same.
(debug_value_range): Same.
(vrp_prop::check_array_ref): Same.
(vrp_prop::check_mem_ref): Same.
(vrp_prop::vrp_initialize): Same.
(vrp_prop::visit_stmt): Same.
(intersect_ranges): Same.
(vrp_prop::visit_phi): Same.
(vrp_prop::vrp_finalize): Same.
(determine_value_range_1): Same.
(determine_value_range): Same.
(vrp_intersect_ranges_1): Rename to...
(vrp_intersect_1): this.
(vrp_intersect_ranges): Rename to...
(value_range::intersect_helper): ...this.
(vrp_meet_1): Rename to...
(value_range::union_helper): ...this.
(vrp_meet): Rename to...
(value_range::union_): ...this.
(copy_value_range): Remove.
* tree-vrp.h (struct value_range): Rewrite into a proper class.
(value_range::vrtype): New.
(value_range::type): New.
(value_range::equiv): New.
(value_range::min): New.
(value_range::max): New.
(value_range::varying_p): New.
(value_range::undefined_p): New.
(value_range::null_p): New.
(value_range::equiv_add): New.
(copy_value_range): Remove.
From-SVN: r265241
|
|
As promised at Cauldron, this patch uses %T and %G with dump_printf and
dump_printf_loc calls to eliminate calls to
dump_generic_expr (MSG_*, arg, TDF_SLIM) (via %T)
and
dump_gimple_stmt (MSG_*, TDF_SLIM, stmt, 0) (via %G)
throughout the middle-end, simplifying numerous dump callsites.
A few calls to these functions didn't match the above pattern; I didn't
touch these. I wasn't able to use %E anywhere.
gcc/ChangeLog:
* tree-data-ref.c (runtime_alias_check_p): Use formatted printing
with %T in place of calls to dump_generic_expr.
(prune_runtime_alias_test_list): Likewise.
(create_runtime_alias_checks): Likewise.
* tree-vect-data-refs.c (vect_check_nonzero_value): Likewise.
(vect_analyze_data_ref_dependence): Likewise.
(vect_slp_analyze_data_ref_dependence): Likewise.
(vect_record_base_alignment): Likewise. Use %G in place of call
to dump_gimple_stmt.
(vect_compute_data_ref_alignment): Likewise.
(verify_data_ref_alignment): Likewise.
(vect_find_same_alignment_drs): Likewise.
(vect_analyze_group_access_1): Likewise.
(vect_analyze_data_ref_accesses): Likewise.
(dependence_distance_ge_vf): Likewise.
(dump_lower_bound): Likewise.
(vect_prune_runtime_alias_test_list): Likewise.
(vect_find_stmt_data_reference): Likewise.
(vect_analyze_data_refs): Likewise.
(vect_create_addr_base_for_vector_ref): Likewise.
(vect_create_data_ref_ptr): Likewise.
* tree-vect-loop-manip.c (vect_set_loop_condition): Likewise.
(vect_can_advance_ivs_p): Likewise.
(vect_update_ivs_after_vectorizer): Likewise.
(vect_gen_prolog_loop_niters): Likewise.
(vect_prepare_for_masked_peels): Likewise.
* tree-vect-loop.c (vect_determine_vf_for_stmt): Likewise.
(vect_determine_vectorization_factor): Likewise.
(vect_is_simple_iv_evolution): Likewise.
(vect_analyze_scalar_cycles_1): Likewise.
(vect_analyze_loop_operations): Likewise.
(report_vect_op): Likewise.
(vect_is_slp_reduction): Likewise.
(check_reduction_path): Likewise.
(vect_is_simple_reduction): Likewise.
(vect_create_epilog_for_reduction): Likewise.
(vect_finalize_reduction:): Likewise.
(vectorizable_induction): Likewise.
(vect_transform_loop_stmt): Likewise.
(vect_transform_loop): Likewise.
(optimize_mask_stores): Likewise.
* tree-vect-patterns.c (vect_pattern_detected): Likewise.
(vect_split_statement): Likewise.
(vect_recog_over_widening_pattern): Likewise.
(vect_recog_average_pattern): Likewise.
(vect_determine_min_output_precision_1): Likewise.
(vect_determine_precisions_from_range): Likewise.
(vect_determine_precisions_from_users): Likewise.
(vect_mark_pattern_stmts): Likewise.
(vect_pattern_recog_1): Likewise.
* tree-vect-slp.c (vect_get_and_check_slp_defs): Likewise.
(vect_record_max_nunits): Likewise.
(vect_build_slp_tree_1): Likewise.
(vect_build_slp_tree_2): Likewise.
(vect_print_slp_tree): Likewise.
(vect_analyze_slp_instance): Likewise.
(vect_detect_hybrid_slp_stmts): Likewise.
(vect_detect_hybrid_slp_1): Likewise.
(vect_slp_analyze_operations): Likewise.
(vect_slp_analyze_bb_1): Likewise.
(vect_transform_slp_perm_load): Likewise.
(vect_schedule_slp_instance): Likewise.
* tree-vect-stmts.c (vect_mark_relevant): Likewise.
(vect_mark_stmts_to_be_vectorized): Likewise.
(vect_init_vector_1): Likewise.
(vect_get_vec_def_for_operand): Likewise.
(vect_finish_stmt_generation_1): Likewise.
(vect_check_load_store_mask): Likewise.
(vectorizable_call): Likewise.
(vectorizable_conversion): Likewise.
(vectorizable_operation): Likewise.
(vectorizable_load): Likewise.
(vect_analyze_stmt): Likewise.
(vect_is_simple_use): Likewise.
(vect_get_vector_types_for_stmt): Likewise.
(vect_get_mask_type_for_stmt): Likewise.
* tree-vectorizer.c (increase_alignment): Likewise.
From-SVN: r264424
|
|
This PR is a wrong-code bug caused by the over-widening support.
The minimum input precisions for a COND_EXPR are supposed to apply
only to the "then" and "else" values, but here we were applying
them to the operands of a nested COND_EXPR comparison instead.
2018-08-01 Richard Sandiford <richard.sandiford@arm.com>
gcc/
PR tree-optimization/86749
* tree-vect-patterns.c (vect_determine_min_output_precision_1):
If the lhs is used in a COND_EXPR, check that it is being used
as the "then" or "else" value.
gcc/testsuite/
PR tree-optimization/86749
* gcc.dg/vect/pr86749.c: New test.
From-SVN: r263213
|
|
This patch removes the stmt_vec_info wrapper class added near the
beginning of the series and turns stmt_vec_info back into a typedef.
2018-07-31 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vectorizer.h (stmt_vec_info): Turn back into a typedef.
(NULL_STMT_VEC_INFO): Delete.
(stmt_vec_info::operator*): Likewise.
(stmt_vec_info::operator gimple *): Likewise.
* tree-vect-loop.c (vectorizable_reduction): Use NULL instead
of NULL_STMT_VEC_INFO.
* tree-vect-patterns.c (vect_init_pattern_stmt): Likewise.
(vect_reassociating_reduction_p): Likewise.
* tree-vect-stmts.c (vect_build_gather_load_calls): Likewise.
(vectorizable_store): Likewise.
* tree-vectorizer.c (vec_info::set_vinfo_for_stmt): Likewise.
(vec_info::free_stmt_vec_infos): Likewise.
From-SVN: r263161
|
|
This patch makes dr_aux link back to both the scalar data_reference
and the containing stmt_vec_info, so that it becomes a suitable key
for a vectorisable reference.
The data_reference link is just STMT_VINFO_DATA_REF, moved here
from _stmt_vec_info. The stmt pointer is a new field and always
tracks the current stmt_vec_info for the reference (which might
be a pattern stmt or the original stmt).
The patch also makes the dr_aux in this current stmt be the one
that counts, rather than have the information stay with the DR_STMT.
A new macro (STMT_VINFO_DR_INFO) gives this information for a given
stmt_info.
In future we could make dr_aux a separate structure, which would
be useful if we want to support multiple data references per stmt.
That seems too much of a diversion for this series though.
2018-07-31 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vectorizer.h (vec_info::move_dr): New member function.
(dataref_aux): Rename to...
(dr_vec_info): ...this and add "dr" and "stmt" fields.
(_stmt_vec_info::dr_aux): Update accordingly.
(_stmt_vec_info::data_ref_info): Delete.
(STMT_VINFO_GROUPED_ACCESS, DR_GROUP_FIRST_ELEMENT)
(DR_GROUP_NEXT_ELEMENT, DR_GROUP_SIZE, DR_GROUP_STORE_COUNT)
(DR_GROUP_GAP, DR_GROUP_SAME_DR_STMT, REDUC_GROUP_FIRST_ELEMENT):
(REDUC_GROUP_NEXT_ELEMENT, REDUC_GROUP_SIZE): Use dr_aux.dr instead
of data_ref.
(STMT_VINFO_DATA_REF): Likewise. Turn into an lvalue.
(STMT_VINFO_DR_INFO): New macro.
(DR_VECT_AUX): Use STMT_VINFO_DR_INKFO and vect_dr_stmt.
(set_dr_misalignment): Update after rename of dataref_aux.
(vect_dr_stmt): Move earlier in file. Return dr_aux.stmt.
* tree-vect-stmts.c (new_stmt_vec_info): Remove redundant
initialization of STMT_VINFO_DATA_REF.
* tree-vectorizer.c (vec_info::move_dr): New function.
* tree-vect-patterns.c (vect_recog_bool_pattern)
(vect_recog_mask_conversion_pattern)
(vect_recog_gather_scatter_pattern): Use it.
* tree-vect-data-refs.c (vect_analyze_data_refs): Initialize
the "dr" and "stmt" fields of dr_vec_info instead of
STMT_VINFO_DATA_REF.
From-SVN: r263152
|
|
This patch adds a pattern_stmt_p field to stmt_vec_info, so that it's
possible to tell whether the statement is a pattern statement without
referring to other statements. The new field goes in what was
previously a hole in the structure, so the size is the same as before.
2018-07-31 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vectorizer.h (_stmt_vec_info::pattern_stmt_p): New field.
(is_pattern_stmt_p): Use it.
* tree-vect-patterns.c (vect_init_pattern_stmt): Set pattern_stmt_p
on pattern statements.
From-SVN: r263151
|
|
vect_pattern_recog_1 took a gimple_stmt_iterator as argument, but was
only interested in the gsi_stmt, not anything else. This patch makes
the associated routines operate directly on stmt_vec_infos.
2018-07-31 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vect-patterns.c (vect_mark_pattern_stmts): Take the
original stmt as a stmt_vec_info rather than a gimple stmt.
(vect_pattern_recog_1): Take the statement directly as a
stmt_vec_info, rather than via a gimple_stmt_iterator.
Update call to vect_mark_pattern_stmts.
(vect_pattern_recog): Update calls accordingly.
From-SVN: r263150
|
|
This second part handles the mechanical change from a gimple stmt
argument to a stmt_vec_info argument. It updates the function
comments if they referred to the argument by name, but it doesn't
try to retrofit mentions to other functions.
2018-07-31 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vectorizer.h (nested_in_vect_loop_p): Move further down
file and take a stmt_vec_info instead of a gimple stmt.
(supportable_widening_operation, vect_finish_replace_stmt)
(vect_finish_stmt_generation, vect_get_store_rhs)
(vect_get_vec_def_for_operand_1, vect_get_vec_def_for_operand)
(vect_get_vec_defs, vect_init_vector, vect_transform_stmt)
(vect_remove_stores, vect_analyze_stmt, vectorizable_condition)
(vect_get_smallest_scalar_type, vect_check_gather_scatter)
(vect_create_data_ref_ptr, bump_vector_ptr)
(vect_permute_store_chain, vect_setup_realignment)
(vect_transform_grouped_load, vect_record_grouped_load_vectors)
(vect_create_addr_base_for_vector_ref, vectorizable_live_operation)
(vectorizable_reduction, vectorizable_induction)
(get_initial_def_for_reduction, is_simple_and_all_uses_invariant)
(vect_get_place_in_interleaving_chain): Take stmt_vec_infos rather
than gimple stmts as arguments.
* tree-vect-data-refs.c (vect_get_smallest_scalar_type)
(vect_preserves_scalar_order_p, vect_slp_analyze_node_dependences)
(can_group_stmts_p, vect_check_gather_scatter)
(vect_create_addr_base_for_vector_ref, vect_create_data_ref_ptr)
(bump_vector_ptr, vect_permute_store_chain, vect_setup_realignment)
(vect_permute_load_chain, vect_shift_permute_load_chain)
(vect_transform_grouped_load)
(vect_record_grouped_load_vectors): Likewise.
* tree-vect-loop.c (vect_fixup_reduc_chain)
(get_initial_def_for_reduction, vect_create_epilog_for_reduction)
(vectorize_fold_left_reduction, is_nonwrapping_integer_induction)
(vectorizable_reduction, vectorizable_induction)
(vectorizable_live_operation, vect_loop_kill_debug_uses): Likewise.
* tree-vect-patterns.c (type_conversion_p, adjust_bool_stmts)
(vect_get_load_store_mask): Likewise.
* tree-vect-slp.c (vect_get_place_in_interleaving_chain)
(vect_analyze_slp_instance, vect_mask_constant_operand_p): Likewise.
* tree-vect-stmts.c (vect_mark_relevant)
(is_simple_and_all_uses_invariant)
(exist_non_indexing_operands_for_use_p, process_use)
(vect_init_vector_1, vect_init_vector, vect_get_vec_def_for_operand_1)
(vect_get_vec_def_for_operand, vect_get_vec_defs)
(vect_finish_stmt_generation_1, vect_finish_replace_stmt)
(vect_finish_stmt_generation, vect_truncate_gather_scatter_offset)
(compare_step_with_zero, vect_get_store_rhs, get_group_load_store_type)
(get_negative_load_store_type, get_load_store_type)
(vect_check_load_store_mask, vect_check_store_rhs)
(vect_build_gather_load_calls, vect_get_strided_load_store_ops)
(vectorizable_bswap, vectorizable_call, vectorizable_simd_clone_call)
(vect_create_vectorized_demotion_stmts, vectorizable_conversion)
(vectorizable_assignment, vectorizable_shift, vectorizable_operation)
(get_group_alias_ptr_type, vectorizable_store, hoist_defs_of_uses)
(vectorizable_load, vectorizable_condition, vectorizable_comparison)
(vect_analyze_stmt, vect_transform_stmt, vect_remove_stores)
(supportable_widening_operation): Likewise.
From-SVN: r263147
|
|
This first part makes functions use stmt_vec_infos instead of
gimple stmts in cases where the stmt_vec_info was already available
and where the change is mechanical. Most of it is just replacing
"stmt" with "stmt_info".
2018-07-31 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vect-data-refs.c (vect_slp_analyze_node_dependences):
(vect_check_gather_scatter, vect_create_data_ref_ptr, bump_vector_ptr)
(vect_permute_store_chain, vect_setup_realignment)
(vect_permute_load_chain, vect_shift_permute_load_chain)
(vect_transform_grouped_load): Use stmt_vec_info rather than gimple
stmts internally, and when passing values to other vectorizer routines.
* tree-vect-loop-manip.c (vect_can_advance_ivs_p): Likewise.
* tree-vect-loop.c (vect_analyze_scalar_cycles_1)
(vect_analyze_loop_operations, get_initial_def_for_reduction)
(vect_create_epilog_for_reduction, vectorize_fold_left_reduction)
(vectorizable_reduction, vectorizable_induction)
(vectorizable_live_operation, vect_transform_loop_stmt)
(vect_transform_loop): Likewise.
* tree-vect-patterns.c (vect_reassociating_reduction_p)
(vect_recog_widen_op_pattern, vect_recog_mixed_size_cond_pattern)
(vect_recog_bool_pattern, vect_recog_gather_scatter_pattern): Likewise.
* tree-vect-slp.c (vect_analyze_slp_instance): Likewise.
(vect_slp_analyze_node_operations_1): Likewise.
* tree-vect-stmts.c (vect_mark_relevant, process_use)
(exist_non_indexing_operands_for_use_p, vect_init_vector_1)
(vect_mark_stmts_to_be_vectorized, vect_get_vec_def_for_operand)
(vect_finish_stmt_generation_1, get_group_load_store_type)
(get_load_store_type, vect_build_gather_load_calls)
(vectorizable_bswap, vectorizable_call, vectorizable_simd_clone_call)
(vect_create_vectorized_demotion_stmts, vectorizable_conversion)
(vectorizable_assignment, vectorizable_shift, vectorizable_operation)
(vectorizable_store, vectorizable_load, vectorizable_condition)
(vectorizable_comparison, vect_analyze_stmt, vect_transform_stmt)
(supportable_widening_operation): Likewise.
(vect_get_vector_types_for_stmt): Likewise.
* tree-vectorizer.h (vect_dr_behavior): Likewise.
From-SVN: r263143
|
|
Various places called vect_dr_stmt or vinfo_for_stmt multiple times
on the same input. This patch makes them reuse the earlier result.
It also splits a couple of single vinfo_for_stmt calls out into
separate statements so that they can be reused in later patches.
2018-07-31 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vect-data-refs.c (vect_analyze_data_ref_dependence)
(vect_slp_analyze_node_dependences, vect_analyze_data_ref_accesses)
(vect_permute_store_chain, vect_permute_load_chain)
(vect_shift_permute_load_chain, vect_transform_grouped_load): Avoid
repeated stmt_vec_info lookups.
* tree-vect-loop-manip.c (vect_can_advance_ivs_p): Likewise.
(vect_update_ivs_after_vectorizer): Likewise.
* tree-vect-loop.c (vect_is_simple_reduction): Likewise.
(vect_create_epilog_for_reduction, vectorizable_reduction): Likewise.
* tree-vect-patterns.c (adjust_bool_stmts): Likewise.
* tree-vect-slp.c (vect_analyze_slp_instance): Likewise.
(vect_bb_slp_scalar_cost): Likewise.
* tree-vect-stmts.c (get_group_alias_ptr_type): Likewise.
From-SVN: r263142
|
|
This patch changes {REDUC,DR}_GROUP_{FIRST,NEXT} element from a
gimple stmt to stmt_vec_info.
2018-07-31 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vectorizer.h (_stmt_vec_info::first_element): Change from
a gimple stmt to a stmt_vec_info.
(_stmt_vec_info::next_element): Likewise.
* tree-vect-data-refs.c (vect_update_misalignment_for_peel)
(vect_slp_analyze_and_verify_node_alignment)
(vect_analyze_group_access_1, vect_analyze_group_access)
(vect_small_gap_p, vect_prune_runtime_alias_test_list)
(vect_create_data_ref_ptr, vect_record_grouped_load_vectors)
(vect_supportable_dr_alignment): Update accordingly.
* tree-vect-loop.c (vect_fixup_reduc_chain): Likewise.
(vect_fixup_scalar_cycles_with_patterns, vect_is_slp_reduction)
(vect_is_simple_reduction, vectorizable_reduction): Likewise.
* tree-vect-patterns.c (vect_reassociating_reduction_p): Likewise.
* tree-vect-slp.c (vect_build_slp_tree_1)
(vect_attempt_slp_rearrange_stmts, vect_supported_load_permutation_p)
(vect_split_slp_store_group, vect_analyze_slp_instance)
(vect_analyze_slp, vect_transform_slp_perm_load): Likewise.
* tree-vect-stmts.c (vect_model_store_cost, vect_model_load_cost)
(get_group_load_store_type, get_load_store_type)
(get_group_alias_ptr_type, vectorizable_store, vectorizable_load)
(vect_transform_stmt, vect_remove_stores): Likewise.
From-SVN: r263135
|
|
This patch changes LOOP_VINFO_REDUCTIONS from an auto_vec<gimple *>
to an auto_vec<stmt_vec_info>. It also changes the associated
vect_force_simple_reduction so that it takes and returns stmt_vec_infos
instead of gimple stmts.
2018-07-31 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vectorizer.h (_loop_vec_info::reductions): Change from an
auto_vec<gimple *> to an auto_vec<stmt_vec_info>.
(vect_force_simple_reduction): Take and return stmt_vec_infos rather
than gimple stmts.
* tree-parloops.c (valid_reduction_p): Take a stmt_vec_info instead
of a gimple stmt.
(gather_scalar_reductions): Update after above interface changes.
* tree-vect-loop.c (vect_analyze_scalar_cycles_1): Likewise.
(vect_is_simple_reduction): Take and return stmt_vec_infos rather
than gimple stmts.
(vect_force_simple_reduction): Likewise.
* tree-vect-patterns.c (vect_pattern_recog_1): Update use of
LOOP_VINFO_REDUCTIONS.
* tree-vect-slp.c (vect_analyze_slp_instance): Likewise.
From-SVN: r263132
|
|
This patch changes STMT_VINFO_RELATED_STMT from a gimple stmt to a
stmt_vec_info.
2018-07-31 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vectorizer.h (_stmt_vec_info::related_stmt): Change from
a gimple stmt to a stmt_vec_info.
(is_pattern_stmt_p): Update accordingly.
* tree-vect-data-refs.c (vect_preserves_scalar_order_p): Likewise.
(vect_record_grouped_load_vectors): Likewise.
* tree-vect-loop.c (vect_determine_vf_for_stmt): Likewise.
(vect_fixup_reduc_chain, vect_update_vf_for_slp): Likewise.
(vect_model_reduction_cost): Likewise.
(vect_create_epilog_for_reduction): Likewise.
(vectorizable_reduction, vectorizable_induction): Likewise.
* tree-vect-patterns.c (vect_init_pattern_stmt): Likewise.
Return the stmt_vec_info for the pattern statement.
(vect_set_pattern_stmt): Update use of STMT_VINFO_RELATED_STMT.
(vect_split_statement, vect_mark_pattern_stmts): Likewise.
* tree-vect-slp.c (vect_detect_hybrid_slp_stmts): Likewise.
(vect_detect_hybrid_slp, vect_get_slp_defs): Likewise.
* tree-vect-stmts.c (vect_mark_relevant): Likewise.
(vect_get_vec_def_for_operand_1, vectorizable_call): Likewise.
(vectorizable_simd_clone_call, vect_analyze_stmt, new_stmt_vec_info)
(free_stmt_vec_info, vect_is_simple_use): Likewise.
From-SVN: r263128
|
|
This patch makes vect_is_simple_use pass back a stmt_vec_info to
those callers that want it. Most users only need the stmt_vec_info
but some need the gimple stmt too.
It's probably high time we added a class to represent "simple operands"
instead, but I have a separate series that tries to clean up how
operands are handled (with a view to allowing mixed vector sizes).
2018-07-31 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vectorizer.h (vect_is_simple_use): Add an optional
stmt_vec_info * parameter before the optional gimple **.
* tree-vect-stmts.c (vect_is_simple_use): Likewise.
(process_use, vect_get_vec_def_for_operand_1): Update callers.
(vect_get_vec_def_for_operand, vectorizable_shift): Likewise.
* tree-vect-loop.c (vectorizable_reduction): Likewise.
(vectorizable_live_operation): Likewise.
* tree-vect-patterns.c (type_conversion_p): Likewise.
(vect_look_through_possible_promotion): Likewise.
(vect_recog_rotate_pattern): Likewise.
* tree-vect-slp.c (vect_get_and_check_slp_defs): Likewise.
From-SVN: r263126
|
|
This patch turns stmt_vec_info into an unspeakably bad wrapper class
and adds an implicit conversion to the associated gimple stmt.
Having this conversion makes the rest of the series easier to write,
but since the class goes away again at the end of the series, I've
not bothered adding any comments or tried to make it pretty.
2018-07-31 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vectorizer.h (stmt_vec_info): Temporarily change from
a typedef to a wrapper class.
(NULL_STMT_VEC_INFO): New macro.
(vec_info::stmt_infos): Change to vec<stmt_vec_info>.
(stmt_vec_info::operator*): New function.
(stmt_vec_info::operator gimple *): Likewise.
(set_vinfo_for_stmt): Use NULL_STMT_VEC_INFO.
(add_stmt_costs): Likewise.
* tree-vect-loop-manip.c (iv_phi_p): Likewise.
* tree-vect-loop.c (vect_compute_single_scalar_iteration_cost)
(vect_get_known_peeling_cost): Likewise.
(vect_estimate_min_profitable_iters): Likewise.
* tree-vect-patterns.c (vect_init_pattern_stmt): Likewise.
* tree-vect-slp.c (vect_remove_slp_scalar_calls): Likewise.
* tree-vect-stmts.c (vect_build_gather_load_calls): Likewise.
(vectorizable_store, free_stmt_vec_infos): Likewise.
(new_stmt_vec_info): Change return type of xcalloc to
_stmt_vec_info *.
From-SVN: r263125
|
|
This patch adds a vec_info helper for checking whether an operand is an
SSA_NAME that is defined in the vectorisable region.
2018-07-31 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vectorizer.h (vec_info::lookup_def): Declare.
* tree-vectorizer.c (vec_info::lookup_def): New function.
* tree-vect-patterns.c (vect_get_internal_def): Use it.
(vect_widened_op_tree): Likewise.
* tree-vect-stmts.c (vect_is_simple_use): Likewise.
* tree-vect-loop.c (vect_analyze_loop_operations): Likewise.
(vectorizable_reduction): Likewise.
(vect_valid_reduction_input_p): Take a stmt_vec_info instead
of a gimple *.
(vect_is_slp_reduction): Update calls accordingly. Use
vec_info::lookup_def.
(vect_is_simple_reduction): Likewise
* tree-vect-slp.c (vect_detect_hybrid_slp_1): Use vec_info::lookup_def.
From-SVN: r263123
|
|
This patch adds a vec_info replacement for vinfo_for_stmt. The main
difference is that the new routine can cope with arbitrary statements,
so there's no need to call vect_stmt_in_region_p first.
The patch only converts calls that are still needed at the end of the
series. Later patches get rid of most other calls to vinfo_for_stmt.
2018-07-31 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vectorizer.h (vec_info::lookup_stmt): Declare.
* tree-vectorizer.c (vec_info::lookup_stmt): New function.
* tree-vect-loop.c (vect_determine_vf_for_stmt): Use it instead
of vinfo_for_stmt.
(vect_determine_vectorization_factor, vect_analyze_scalar_cycles_1)
(vect_compute_single_scalar_iteration_cost, vect_analyze_loop_form)
(vect_update_vf_for_slp, vect_analyze_loop_operations)
(vect_is_slp_reduction, vectorizable_induction)
(vect_transform_loop_stmt, vect_transform_loop): Likewise.
* tree-vect-patterns.c (vect_init_pattern_stmt):
(vect_determine_min_output_precision_1, vect_determine_precisions)
(vect_pattern_recog): Likewise.
* tree-vect-stmts.c (vect_analyze_stmt, vect_transform_stmt): Likewise.
* config/powerpcspe/powerpcspe.c (rs6000_density_test): Likewise.
* config/rs6000/rs6000.c (rs6000_density_test): Likewise.
* tree-vect-slp.c (vect_detect_hybrid_slp_stmts): Likewise.
(vect_detect_hybrid_slp_1, vect_detect_hybrid_slp_2)
(vect_detect_hybrid_slp): Likewise. Change the walk_stmt_info
info field from a loop to a loop_vec_info.
From-SVN: r263122
|
|
This patch adds a vec_info function for allocating and setting
stmt_vec_infos. It's the start of a long process of removing
the global stmt_vec_info array.
2018-07-31 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vectorizer.h (stmt_vec_info): Move typedef earlier in file.
(vec_info::add_stmt): Declare.
* tree-vectorizer.c (vec_info::add_stmt): New function.
* tree-vect-data-refs.c (vect_create_data_ref_ptr): Use it.
* tree-vect-loop.c (_loop_vec_info::_loop_vec_info): Likewise.
(vect_create_epilog_for_reduction, vectorizable_reduction): Likewise.
(vectorizable_induction): Likewise.
* tree-vect-slp.c (_bb_vec_info::_bb_vec_info): Likewise.
* tree-vect-stmts.c (vect_finish_stmt_generation_1): Likewise.
(vectorizable_simd_clone_call, vectorizable_store): Likewise.
(vectorizable_load): Likewise.
* tree-vect-patterns.c (vect_init_pattern_stmt): Likewise.
(vect_recog_bool_pattern, vect_recog_mask_conversion_pattern)
(vect_recog_gather_scatter_pattern): Likewise.
(append_pattern_def_seq): Likewise. Remove a check that is
performed by add_stmt itself.
From-SVN: r263121
|
|
2018-07-18 Richard Biener <rguenther@suse.de>
PR tree-optimization/86557
* tree-vect-patterns.c (vect_recog_divmod_pattern): Also handle
EXACT_DIV_EXPR.
From-SVN: r262854
|
|
This patch uses IFN_COND_* to vectorise conditionally-executed,
potentially-trapping arithmetic, such as most floating-point
ops with -ftrapping-math. E.g.:
if (cond) { ... x = a + b; ... }
becomes:
...
x = .COND_ADD (cond, a, b, else_value);
...
When this transformation is done on its own, the value of x for
!cond isn't important, so else_value is simply the target's
preferred_else_value (i.e. the value it can handle the most
efficiently).
However, the patch also looks for the equivalent of:
y = cond ? x : c;
in which the "then" value is the result of the conditionally-executed
operation and the "else" value "c" is some value that is available at x.
In that case we can instead use:
x = .COND_ADD (cond, a, b, c);
and replace uses of y with uses of x.
The patch also looks for:
y = !cond ? c : x;
which can be transformed in the same way. This involved adding a new
utility function inverse_conditions_p, which was already open-coded
in a more limited way in match.pd.
2018-07-12 Richard Sandiford <richard.sandiford@linaro.org>
gcc/
* fold-const.h (inverse_conditions_p): Declare.
* fold-const.c (inverse_conditions_p): New function.
* match.pd: Use inverse_conditions_p. Add folds of view_converts
that test the inverse condition of a conditional internal function.
* internal-fn.h (vectorized_internal_fn_supported_p): Declare.
* internal-fn.c (internal_fn_mask_index): Handle conditional
internal functions.
(vectorized_internal_fn_supported_p): New function.
* tree-if-conv.c: Include internal-fn.h and fold-const.h.
(any_pred_load_store): Replace with...
(need_to_predicate): ...this new variable.
(redundant_ssa_names): New variable.
(ifcvt_can_use_mask_load_store): Move initial checks to...
(ifcvt_can_predicate): ...this new function. Handle tree codes
for which a conditional internal function exists.
(if_convertible_gimple_assign_stmt_p): Use ifcvt_can_predicate
instead of ifcvt_can_use_mask_load_store. Update after variable
name change.
(predicate_load_or_store): New function, split out from
predicate_mem_writes.
(check_redundant_cond_expr): New function.
(value_available_p): Likewise.
(predicate_rhs_code): Likewise.
(predicate_mem_writes): Rename to...
(predicate_statements): ...this. Use predicate_load_or_store
and predicate_rhs_code.
(combine_blocks, tree_if_conversion): Update after above name changes.
(ifcvt_local_dce): Handle redundant_ssa_names.
* tree-vect-patterns.c (vect_recog_mask_conversion_pattern): Handle
general conditional functions.
* tree-vect-stmts.c (vectorizable_call): Likewise.
gcc/testsuite/
* gcc.dg/vect/vect-cond-arith-4.c: New test.
* gcc.dg/vect/vect-cond-arith-5.c: Likewise.
* gcc.target/aarch64/sve/cond_arith_1.c: Likewise.
* gcc.target/aarch64/sve/cond_arith_1_run.c: Likewise.
* gcc.target/aarch64/sve/cond_arith_2.c: Likewise.
* gcc.target/aarch64/sve/cond_arith_2_run.c: Likewise.
* gcc.target/aarch64/sve/cond_arith_3.c: Likewise.
* gcc.target/aarch64/sve/cond_arith_3_run.c: Likewise.
From-SVN: r262589
|
|
The PR85694 series added a vectype argument to append_pattern_def_seq.
This patch makes more callers use it.
2018-07-03 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vect-patterns.c (vect_recog_rotate_pattern)
(vect_recog_vector_vector_shift_pattern, vect_recog_divmod_pattern)
(vect_recog_mixed_size_cond_pattern, adjust_bool_pattern_cast)
(adjust_bool_pattern, vect_recog_bool_pattern): Pass the vector
type to append_pattern_def_seq instead of creating a stmt_vec_info
directly.
(build_mask_conversion): Likewise. Remove vinfo argument.
(vect_add_conversion_to_patterm): Likewise, renaming to...
(vect_add_conversion_to_pattern): ...this.
(vect_recog_mask_conversion_pattern): Update call to
build_mask_conversion. Pass the vector type to
append_pattern_def_seq here too.
(vect_recog_gather_scatter_pattern): Update call to
vect_add_conversion_to_pattern.
From-SVN: r262338
|
|
Various recognisers set PATTERN_DEF_SEQ to null before adding
statements to it, but it should always be null at that point anyway.
This patch asserts for that in vect_pattern_recog_1 and removes
the redundant code.
2018-07-03 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vect-patterns.c (new_pattern_def_seq): Delete.
(vect_recog_dot_prod_pattern, vect_recog_sad_pattern)
(vect_recog_widen_op_pattern, vect_recog_over_widening_pattern)
(vect_recog_rotate_pattern, vect_synth_mult_by_constant): Don't set
STMT_VINFO_PATTERN_DEF_SEQ to null here.
(vect_recog_pow_pattern, vect_recog_vector_vector_shift_pattern)
(vect_recog_mixed_size_cond_pattern, vect_recog_bool_pattern): Use
append_pattern_def_seq instead of new_pattern_def_seq.
(vect_recog_divmod_pattern): Do both of the above.
(vect_pattern_recog_1): Assert that STMT_VINO_PATTERN_DEF_SEQ
is null.
From-SVN: r262337
|
|
The PR85694 series removed the only cases in which a pattern recogniser
could attach patterns to more than one statement. I think it would be
better to avoid adding any new instances of that, since it interferes
with the normal matching order.
This patch therefore switches the interface back to passing a single
statement instead of a vector. It also gets rid of the clearing of
STMT_VINFO_RELATED_STMT on failure, since no recognisers use it now.
2018-07-03 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vect-patterns.c (vect_recog_dot_prod_pattern):
(vect_recog_sad_pattern, vect_recog_widen_op_pattern)
(vect_recog_widen_mult_pattern, vect_recog_pow_pattern):
(vect_recog_widen_sum_pattern, vect_recog_over_widening_pattern)
(vect_recog_average_pattern, vect_recog_cast_forwprop_pattern)
(vect_recog_widen_shift_pattern, vect_recog_rotate_pattern)
(vect_recog_vector_vector_shift_pattern, vect_synth_mult_by_constant)
(vect_recog_mult_pattern, vect_recog_divmod_pattern)
(vect_recog_mixed_size_cond_pattern, vect_recog_bool_pattern)
(vect_recog_mask_conversion_pattern): Replace vec<gimple *>
parameter with a single stmt_vec_info.
(vect_recog_func_ptr): Likewise.
(vect_recog_gather_scatter_pattern): Likewise, folding in...
(vect_try_gather_scatter_pattern): ...this.
(vect_pattern_recog_1): Remove stmts_to_replace and just pass
the stmt_vec_info of the statement to be matched. Don't clear
STMT_VINFO_RELATED_STMT.
(vect_pattern_recog): Update call accordingly.
From-SVN: r262336
|
|
This patch adds detection of average instructions:
a = (((wide) b + (wide) c) >> 1);
--> a = (wide) .AVG_FLOOR (b, c);
a = (((wide) b + (wide) c + 1) >> 1);
--> a = (wide) .AVG_CEIL (b, c);
in cases where users of "a" need only the low half of the result,
making the cast to (wide) redundant. The heavy lifting was done by
earlier patches.
This showed up another problem in vectorizable_call: if the call is a
pattern definition statement rather than the main pattern statement,
the type of vectorised call might be different from the type of the
original statement.
2018-07-03 Richard Sandiford <richard.sandiford@arm.com>
gcc/
PR tree-optimization/85694
* doc/md.texi (avgM3_floor, uavgM3_floor, avgM3_ceil)
(uavgM3_ceil): Document new optabs.
* doc/sourcebuild.texi (vect_avg_qi): Document new target selector.
* internal-fn.def (IFN_AVG_FLOOR, IFN_AVG_CEIL): New internal
functions.
* optabs.def (savg_floor_optab, uavg_floor_optab, savg_ceil_optab)
(savg_ceil_optab): New optabs.
* tree-vect-patterns.c (vect_recog_average_pattern): New function.
(vect_vect_recog_func_ptrs): Add it.
* tree-vect-stmts.c (vectorizable_call): Get the type of the zero
constant directly from the associated lhs.
gcc/testsuite/
PR tree-optimization/85694
* lib/target-supports.exp (check_effective_target_vect_avg_qi): New
proc.
* gcc.dg/vect/vect-avg-1.c: New test.
* gcc.dg/vect/vect-avg-2.c: Likewise.
* gcc.dg/vect/vect-avg-3.c: Likewise.
* gcc.dg/vect/vect-avg-4.c: Likewise.
* gcc.dg/vect/vect-avg-5.c: Likewise.
* gcc.dg/vect/vect-avg-6.c: Likewise.
* gcc.dg/vect/vect-avg-7.c: Likewise.
* gcc.dg/vect/vect-avg-8.c: Likewise.
* gcc.dg/vect/vect-avg-9.c: Likewise.
* gcc.dg/vect/vect-avg-10.c: Likewise.
* gcc.dg/vect/vect-avg-11.c: Likewise.
* gcc.dg/vect/vect-avg-12.c: Likewise.
* gcc.dg/vect/vect-avg-13.c: Likewise.
* gcc.dg/vect/vect-avg-14.c: Likewise.
From-SVN: r262335
|
|
The main over-widening patch can introduce quite a few extra casts,
and in many cases those casts simply "tap into" an intermediate
point in an existing extension. E.g. if we have:
unsigned char a;
int ax = (int) a;
and a later operation using ax is shortened to "unsigned short",
we would need:
unsigned short ax' = (unsigned short) a;
The a->ax extension requires one set of unpacks to get to unsigned
short and another set of unpacks to get to int. The first set are
then duplicated for ax'. If both ax and ax' are needed, the a->ax'
extension would end up counting twice during cost calculations.
This patch rewrites the original:
int ax = (int) a;
into a pattern:
unsigned short ax' = (unsigned short) a;
int ax = (int) ax';
so that each extension only counts once.
2018-07-03 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vect-patterns.c (vect_split_statement): New function.
(vect_convert_input): Use it to try to split an existing cast.
gcc/testsuite/
* gcc.dg/vect/vect-over-widen-5.c: Test that the extensions
get split into two for use by the over-widening pattern.
* gcc.dg/vect/vect-over-widen-6.c: Likewise.
* gcc.dg/vect/vect-over-widen-7.c: Likewise.
* gcc.dg/vect/vect-over-widen-8.c: Likewise.
* gcc.dg/vect/vect-over-widen-9.c: Likewise.
* gcc.dg/vect/vect-over-widen-10.c: Likewise.
* gcc.dg/vect/vect-over-widen-11.c: Likewise.
* gcc.dg/vect/vect-over-widen-12.c: Likewise.
* gcc.dg/vect/vect-over-widen-13.c: Likewise.
* gcc.dg/vect/vect-over-widen-14.c: Likewise.
* gcc.dg/vect/vect-over-widen-15.c: Likewise.
* gcc.dg/vect/vect-over-widen-16.c: Likewise.
* gcc.dg/vect/vect-over-widen-22.c: New test.
From-SVN: r262334
|
|
This patch is the main part of PR85694. The aim is to recognise at least:
signed char *a, *b, *c;
...
for (int i = 0; i < 2048; i++)
c[i] = (a[i] + b[i]) >> 1;
as an over-widening pattern, since the addition and shift can be done
on shorts rather than ints. However, it ended up being a lot more
general than that.
The current over-widening pattern detection is limited to a few simple
cases: logical ops with immediate second operands, and shifts by a
constant. These cases are enough for common pixel-format conversion
and can be detected in a peephole way.
The loop above requires two generalisations of the current code: support
for addition as well as logical ops, and support for non-constant second
operands. These are harder to detect in the same peephole way, so the
patch tries to take a more global approach.
The idea is to get information about the minimum operation width
in two ways:
(1) by using the range information attached to the SSA_NAMEs
(effectively a forward walk, since the range info is
context-independent).
(2) by back-propagating the number of output bits required by
users of the result.
As explained in the comments, there's a balance to be struck between
narrowing an individual operation and fitting in with the surrounding
code. The approach is pretty conservative: if we could narrow an
operation to N bits without changing its semantics, it's OK to do that if:
- no operations later in the chain require more than N bits; or
- all internally-defined inputs are extended from N bits or fewer,
and at least one of them is single-use.
See the comments for the rationale.
I didn't bother adding STMT_VINFO_* wrappers for the new fields
since the code seemed more readable without.
2018-06-20 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* poly-int.h (print_hex): New function.
* dumpfile.h (dump_dec, dump_hex): Declare.
* dumpfile.c (dump_dec, dump_hex): New poly_wide_int functions.
* tree-vectorizer.h (_stmt_vec_info): Add min_output_precision,
min_input_precision, operation_precision and operation_sign.
* tree-vect-patterns.c (vect_get_range_info): New function.
(vect_same_loop_or_bb_p, vect_single_imm_use)
(vect_operation_fits_smaller_type): Delete.
(vect_look_through_possible_promotion): Add an optional
single_use_p parameter.
(vect_recog_over_widening_pattern): Rewrite to use new
stmt_vec_info infomration. Handle one operation at a time.
(vect_recog_cast_forwprop_pattern, vect_narrowable_type_p)
(vect_truncatable_operation_p, vect_set_operation_type)
(vect_set_min_input_precision): New functions.
(vect_determine_min_output_precision_1): Likewise.
(vect_determine_min_output_precision): Likewise.
(vect_determine_precisions_from_range): Likewise.
(vect_determine_precisions_from_users): Likewise.
(vect_determine_stmt_precisions, vect_determine_precisions): Likewise.
(vect_vect_recog_func_ptrs): Put over_widening first.
Add cast_forwprop.
(vect_pattern_recog): Call vect_determine_precisions.
gcc/testsuite/
* gcc.dg/vect/vect-widen-mult-u8-u32.c: Check specifically for a
widen_mult pattern.
* gcc.dg/vect/vect-over-widen-1.c: Update the scan tests for new
over-widening messages.
* gcc.dg/vect/vect-over-widen-1-big-array.c: Likewise.
* gcc.dg/vect/vect-over-widen-2.c: Likewise.
* gcc.dg/vect/vect-over-widen-2-big-array.c: Likewise.
* gcc.dg/vect/vect-over-widen-3.c: Likewise.
* gcc.dg/vect/vect-over-widen-3-big-array.c: Likewise.
* gcc.dg/vect/vect-over-widen-4.c: Likewise.
* gcc.dg/vect/vect-over-widen-4-big-array.c: Likewise.
* gcc.dg/vect/bb-slp-over-widen-1.c: New test.
* gcc.dg/vect/bb-slp-over-widen-2.c: Likewise.
* gcc.dg/vect/vect-over-widen-5.c: Likewise.
* gcc.dg/vect/vect-over-widen-6.c: Likewise.
* gcc.dg/vect/vect-over-widen-7.c: Likewise.
* gcc.dg/vect/vect-over-widen-8.c: Likewise.
* gcc.dg/vect/vect-over-widen-9.c: Likewise.
* gcc.dg/vect/vect-over-widen-10.c: Likewise.
* gcc.dg/vect/vect-over-widen-11.c: Likewise.
* gcc.dg/vect/vect-over-widen-12.c: Likewise.
* gcc.dg/vect/vect-over-widen-13.c: Likewise.
* gcc.dg/vect/vect-over-widen-14.c: Likewise.
* gcc.dg/vect/vect-over-widen-15.c: Likewise.
* gcc.dg/vect/vect-over-widen-16.c: Likewise.
* gcc.dg/vect/vect-over-widen-17.c: Likewise.
* gcc.dg/vect/vect-over-widen-18.c: Likewise.
* gcc.dg/vect/vect-over-widen-19.c: Likewise.
* gcc.dg/vect/vect-over-widen-20.c: Likewise.
* gcc.dg/vect/vect-over-widen-21.c: Likewise.
From-SVN: r262333
|
|
r262275 allowed pattern matching on pattern statements. Testing for
SVE on more benchmarks showed a case where this interacted badly
with 14/n.
The new over-widening detection could narrow a COND_EXPR A to another
COND_EXPR B, which mixed_size_cond could then match. This was working
as expected. However, we left B (now dead) in the pattern definition
sequence with a non-null PATTERN_DEF_SEQ. mask_conversion also
matched B, and unlike most recognisers, didn't clear PATTERN_DEF_SEQ
before adding statements to it. This meant that the statements
created by mixed_size_cond appeared in two supposedy separate
sequences, causing much confusion.
This patch removes pattern statements that are replaced by further
pattern statements. As a belt-and-braces fix, it also nullifies
PATTERN_DEF_SEQ on failure, in the same way Richard B. did recently
for RELATED_STMT.
I have patches to clean up the PATTERN_DEF_SEQ handling, but they
only apply after the complete PR85694 sequence, whereas this needs
to go in before 14/n.
2018-07-03 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vect-patterns.c (vect_mark_pattern_stmts): Remove pattern
statements that have been replaced by further pattern statements.
(vect_pattern_recog_1): Clear STMT_VINFO_PATTERN_DEF_SEQ on failure.
gcc/testsuite/
* gcc.dg/vect/vect-mixed-size-cond-1.c: New test.
From-SVN: r262332
|
|
Noticed by Christophe on arm-none-linux-gnueabihf.
2018-07-02 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vect-patterns.c (vect_recog_widen_shift_pattern): Fix typo
in dump string.
From-SVN: r262308
|
|
vect_recog_rotate_pattern had code to prevent operations
on invariants being vectorised unnecessarily:
if (dt == vect_external_def
&& TREE_CODE (oprnd1) == SSA_NAME
&& is_a <loop_vec_info> (vinfo))
{
struct loop *loop = as_a <loop_vec_info> (vinfo)->loop;
ext_def = loop_preheader_edge (loop);
if (!SSA_NAME_IS_DEFAULT_DEF (oprnd1))
{
basic_block bb = gimple_bb (SSA_NAME_DEF_STMT (oprnd1));
if (bb == NULL
|| !dominated_by_p (CDI_DOMINATORS, ext_def->dest, bb))
ext_def = NULL;
}
}
[..]
if (ext_def)
{
basic_block new_bb
= gsi_insert_on_edge_immediate (ext_def, def_stmt);
gcc_assert (!new_bb);
}
This patch reuses the same idea for casts of invariants created
during widening optimisations.
One hitch was that vect_loop_versioning asserted that the vector loop
preheader was still empty, although the cfg transformation it's doing
should be correct either way.
2018-06-30 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vect-patterns.c (vect_get_external_def_edge): New function,
split out from...
(vect_recog_rotate_pattern): ...here.
(vect_convert_input): Try to insert casts of invariants in the
preheader.
* tree-vect-loop-manip.c (vect_loop_versioning): Don't require the
preheader to be empty.
gcc/testsuite/
* gcc.dg/vect/vect-widen-mult-extern-1.c: New test.
From-SVN: r262277
|
|
This patch adds helper functions for detecting widened operations and
generalises the existing code to handle more cases.
One of the main changes is to recognise multi-stage type conversions,
which are possible even in the original IR and can also occur as a
result of earlier pattern matching (especially after the main
over-widening patch). E.g. for:
unsigned int res = 0;
for (__INTPTR_TYPE__ i = 0; i < N; ++i)
{
int av = a[i];
int bv = b[i];
short diff = av - bv;
unsigned short abs = diff < 0 ? -diff : diff;
res += abs;
}
we have:
_9 = _7 - _8;
diff_20 = (short int) _9;
_10 = (int) diff_20;
_11 = ABS_EXPR <_10>;
where the first cast establishes the sign of the promotion done
by the second cast.
vect_recog_sad_pattern didn't handle this kind of intermediate promotion
between the MINUS_EXPR and the ABS_EXPR. Sign extensions and casts from
unsigned to signed are both OK there. Unsigned promotions aren't, and
need to be rejected, but should have been folded away earlier anyway.
Also, the dot_prod and widen_sum patterns both required the promotions
to be from one signedness to the same signedness, rather than say signed
char to unsigned int. That shouldn't be necessary, since it's only the
sign of the input to the promotion that matters. Nothing requires the
narrow and wide types in a DOT_PROD_EXPR or WIDEN_SUM_EXPR to have the
same sign (and IMO that's a good thing).
Fixing these fixed an XFAIL in gcc.dg/vect/vect-widen-mult-sum.c.
vect_widened_op_tree is a bit more general than the current patch needs,
since it copes with a tree of operations rather than a single statement.
This is used by the later average-detection patch.
The patch also uses a common routine to handle both the WIDEN_MULT_EXPR
and WIDEN_LSHIFT_EXPR patterns. I hope this could be reused for other
similar operations in future.
Also, the patch means we recognise the index calculations in
vect-mult-const-pattern*.c as widening multiplications, whereas the
scan test was expecting them to be recognised as mult patterns instead.
The patch makes the tests check specifically for the multiplication we
care about.
2018-06-30 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vect-patterns.c (append_pattern_def_seq): Take an optional
vector type. If given, install it in the new statement's
STMT_VINFO_VECTYPE.
(vect_element_precision): New function.
(vect_unpromoted_value): New struct.
(vect_unpromoted_value::vect_unpromoted_value): New function.
(vect_unpromoted_value::set_op): Likewise.
(vect_look_through_possible_promotion): Likewise.
(vect_joust_widened_integer, vect_joust_widened_type): Likewise.
(vect_widened_op_tree, vect_convert_input): Likewise.
(vect_convert_inputs, vect_convert_output): Likewise.
(vect_recog_dot_prod_pattern): Use vect_look_through_possible_promotion
to handle the optional cast of the multiplication result and
vect_widened_op_tree to detect the widened multiplication itself.
Do not require the input and output of promotion casts to have
the same sign, but base the signedness of the operation on the
input rather than the result. If the pattern includes two
promotions, check that those promotions have the same sign.
Do not restrict the MULT_EXPR handling to a double-width result;
handle quadruple-width results and wider. Use vect_convert_inputs
to convert the inputs to the common type.
(vect_recog_sad_pattern): Use vect_look_through_possible_promotion
to handle the optional cast of the ABS result. Also allow a sign
change or a sign extension between the ABS and MINUS.
Use vect_widened_op_tree to detect the widened subtraction and use
vect_convert_inputs to convert the inputs to the common type.
(vect_handle_widen_op_by_const): Delete.
(vect_recog_widen_op_pattern): New function.
(vect_recog_widen_mult_pattern): Use it.
(vect_recog_widen_shift_pattern): Likewise.
(vect_recog_widen_sum_pattern): Use
vect_look_through_possible_promotion to handle the promoted
PLUS_EXPR operand.
gcc/testsuite/
* gcc.dg/vect/vect-widen-mult-sum.c: Remove xfail.
* gcc.dg/vect/no-scevccp-outer-6.c: Don't match widened multiplications
by 4 in the computation of a[i].
* gcc.dg/vect/vect-mult-const-pattern-1.c: Test specifically for the
main multiplication constant.
* gcc.dg/vect/vect-mult-const-pattern-2.c: Likewise.
* gcc.dg/vect/vect-widen-mult-const-s16.c: Likewise.
* gcc.dg/vect/vect-widen-mult-const-u16.c: Likewise. Expect the
pattern to cast the result to int.
* gcc.dg/vect/vect-reduc-dot-1.c: New test.
* gcc.dg/vect/vect-reduc-dot-2.c: Likewise.
* gcc.dg/vect/vect-reduc-dot-3.c: Likewise.
* gcc.dg/vect/vect-reduc-dot-4.c: Likewise.
* gcc.dg/vect/vect-reduc-dot-5.c: Likewise.
* gcc.dg/vect/vect-reduc-dot-6.c: Likewise.
* gcc.dg/vect/vect-reduc-dot-7.c: Likewise.
* gcc.dg/vect/vect-reduc-dot-8.c: Likewise.
* gcc.dg/vect/vect-reduc-sad-1.c: Likewise.
* gcc.dg/vect/vect-reduc-sad-2.c: Likewise.
* gcc.dg/vect/vect-reduc-sad-3.c: Likewise.
* gcc.dg/vect/vect-reduc-sad-4.c: Likewise.
* gcc.dg/vect/vect-reduc-sad-5.c: Likewise.
* gcc.dg/vect/vect-reduc-sad-6.c: Likewise.
* gcc.dg/vect/vect-reduc-sad-7.c: Likewise.
* gcc.dg/vect/vect-reduc-sad-8.c: Likewise.
* gcc.dg/vect/vect-widen-mult-1.c: Likewise.
* gcc.dg/vect/vect-widen-mult-2.c: Likewise.
* gcc.dg/vect/vect-widen-mult-3.c: Likewise.
* gcc.dg/vect/vect-widen-mult-4.c: Likewise.
From-SVN: r262276
|
|
Although the first pattern match wins in the sense that no later
function can match the *old* gimple statement, it still seems worth
letting them match the *new* gimple statements, just like we would if
the original IR had included that sequence from the outset.
This is mostly true after the later patch for PR85694, where e.g. we
could recognise:
signed char a;
int ap = (int) a;
int res = ap * 3;
as the pattern:
short ap' = (short) a;
short res = ap' * 3; // S1: definition statement
int res = (int) res; // S2: pattern statement
and then apply the mult pattern to "ap' * 3". The patch needs to
come first (without its own test cases) so that the main over-widening
patch doesn't regress anything.
2018-06-30 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* gimple-iterator.c (gsi_for_stmt): Add a new overload that takes
the containing gimple_seq *.
* gimple-iterator.h (gsi_for_stmt): Declare it.
* tree-vect-patterns.c (vect_recog_dot_prod_pattern)
(vect_recog_sad_pattern, vect_recog_widen_sum_pattern)
(vect_recog_widen_shift_pattern, vect_recog_rotate_pattern)
(vect_recog_vector_vector_shift_pattern, vect_recog_divmod_pattern)
(vect_recog_mask_conversion_pattern): Remove STMT_VINFO_IN_PATTERN_P
checks.
(vect_init_pattern_stmt, vect_set_pattern_stmt): New functions,
split out from...
(vect_mark_pattern_stmts): ...here. Handle cases in which the
statement being replaced is part of an existing pattern
definition sequence, inserting the new pattern statements before
the original one.
(vect_pattern_recog_1): Don't return a bool. If the statement
is already part of a pattern, instead apply pattern matching
to the pattern definition statements. Don't clear the
STMT_VINFO_RELATED_STMT if is_pattern_stmt_p.
(vect_pattern_recog): Don't break after the first match;
continue processing the pattern definition statements instead.
Don't bail out for STMT_VINFO_IN_PATTERN_P here.
From-SVN: r262275
|
|
This patch adds an overload of vect_reassociating_reduction_p
that checks for a vectorizable associative reduction,
since the check was duplicated in three functions.
2018-06-30 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vect-patterns.c (vect_reassociating_reduction_p): New function.
(vect_recog_dot_prod_pattern, vect_recog_sad_pattern)
(vect_recog_widen_sum_pattern): Use it.
From-SVN: r262274
|
|
As suggested by Richard B., this patch makes vect_is_simple_use check
whether a defining statement has been replaced by a pattern statement,
and if so returns the pattern statement instead.
The reason for doing this is that the main patch for PR85694
makes over_widening handle more general cases. These over-widened
patterns can still be useful when matching later statements;
e.g. an overwidened MULT_EXPR could be the input to a DOT_PROD_EXPR.
The patch doesn't do anything with the STMT_VINFO_IN_PATTERN_P checks
in vect_recog_over_widening_pattern or vect_recog_widen_shift_pattern
since later patches rewrite them anyway.
Doing this fixed an XFAIL in vect-reduc-dot-u16b.c.
2018-06-30 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vect-loop.c (vectorizable_reduction): Assert that the
phi is not a pattern statement and has not been replaced by
a pattern statement.
* tree-vect-patterns.c (type_conversion_p): Don't check
STMT_VINFO_IN_PATTERN_P.
(vect_recog_vector_vector_shift_pattern): Likewise.
(vect_recog_dot_prod_pattern): Expect vect_is_simple_use to return
the pattern statement rather than the original statement; check
directly for a WIDEN_MULT_EXPR here.
* tree-vect-slp.c (vect_get_and_check_slp_defs): Expect
vect_is_simple_use to return the pattern statement rather
than the original statement; use is_pattern_stmt_p to check
for such a pattern statement.
* tree-vect-stmts.c (process_use): Expect vect_is_simple_use
to return the pattern statement rather than the original statement;
don't do the same transformation here.
(vect_is_simple_use): If the defining statement has been replaced
by a pattern statement, return the pattern statement instead.
Remove the corresponding (local) transformation from the vectype
overload.
gcc/testsuite/
* gcc.dg/vect/vect-reduc-dot-u16b.c: Remove xfail and update the
test for vectorization along the lines described in the comment.
From-SVN: r262273
|
|
As suggested by Richard B., this patch reorders the arguments to
vect_is_simple_use so that def_stmt comes last and is optional.
Many callers can then drop it, making it more obvious which of
the remaining calls would be affected by the next patch.
2018-06-30 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vectorizer.h (vect_is_simple_use): Move the gimple ** to the
end and default to null.
* tree-vect-loop.c (vect_create_epilog_for_reduction)
(vectorizable_reduction): Update calls accordingly, dropping the
gimple ** argument if the passed-back statement isn't needed.
* tree-vect-patterns.c (vect_get_internal_def, type_conversion_p)
(vect_recog_rotate_pattern): Likewise.
(vect_recog_mask_conversion_pattern): Likewise.
* tree-vect-slp.c (vect_get_and_check_slp_defs): Likewise.
(vect_mask_constant_operand_p): Likewise.
* tree-vect-stmts.c (is_simple_and_all_uses_invariant, process_use):
(vect_model_simple_cost, vect_get_vec_def_for_operand): Likewise.
(get_group_load_store_type, get_load_store_type): Likewise.
(vect_check_load_store_mask, vect_check_store_rhs): Likewise.
(vectorizable_call, vectorizable_simd_clone_call): Likewise.
(vectorizable_conversion, vectorizable_assignment): Likewise.
(vectorizable_shift, vectorizable_operation): Likewise.
(vectorizable_store, vect_is_simple_cond): Likewise.
(vectorizable_condition, vectorizable_comparison): Likewise.
(get_same_sized_vectype, vect_get_mask_type_for_stmt): Likewise.
(vect_is_simple_use): Rename the def_stmt argument to def_stmt_out
and move it to the end. Cope with null def_stmt_outs.
From-SVN: r262272
|
|
2018-06-21 Richard Biener <rguenther@suse.de>
* tree-data-ref.c (dr_step_indicator): Handle NULL DR_STEP.
* tree-vect-data-refs.c (vect_analyze_possibly_independent_ddr):
Avoid calling vect_mark_for_runtime_alias_test with gathers or scatters.
(vect_analyze_data_ref_dependence): Re-order checks to deal with
NULL DR_STEP.
(vect_record_base_alignments): Do not record base alignment
for gathers or scatters.
(vect_compute_data_ref_alignment): Drop return value that is always
true. Bail out early for gathers or scatters.
(vect_enhance_data_refs_alignment): Bail out early for gathers
or scatters.
(vect_find_same_alignment_drs): Likewise.
(vect_analyze_data_refs_alignment): Remove dead code.
(vect_slp_analyze_and_verify_node_alignment): Likewise.
(vect_analyze_data_refs): For possible gathers or scatters do
not create an alternate DR, just check their possible validity
and mark them. Adjust DECL_NONALIASED handling to not rely
on DR_BASE_ADDRESS.
* tree-vect-loop-manip.c (vect_update_inits_of_drs): Do not
update inits of gathers or scatters.
* tree-vect-patterns.c (vect_recog_mask_conversion_pattern):
Also copy gather/scatter flag to pattern vinfo.
From-SVN: r261834
|
|
This patch makes pattern recognisers do their own checking for vector
types and target support. Previously some recognisers did this
themselves and some left it to vect_pattern_recog_1.
Doing this means we can get rid of the type_in argument, which was
ignored if the recogniser did its own checking. It also means
we create fewer junk statements.
2018-06-20 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vectorizer.h (NUM_PATTERNS, vect_recog_func_ptr): Move to
tree-vect-patterns.c.
* tree-vect-patterns.c (vect_supportable_direct_optab_p): New function.
(vect_recog_dot_prod_pattern): Use it. Remove the type_in argument.
(vect_recog_sad_pattern): Likewise.
(vect_recog_widen_sum_pattern): Likewise.
(vect_recog_pow_pattern): Likewise. Check for a null vectype.
(vect_recog_widen_shift_pattern): Remove the type_in argument.
(vect_recog_rotate_pattern): Likewise.
(vect_recog_mult_pattern): Likewise.
(vect_recog_vector_vector_shift_pattern): Likewise.
(vect_recog_divmod_pattern): Likewise.
(vect_recog_mixed_size_cond_pattern): Likewise.
(vect_recog_bool_pattern): Likewise.
(vect_recog_mask_conversion_pattern): Likewise.
(vect_try_gather_scatter_pattern): Likewise.
(vect_recog_widen_mult_pattern): Likewise. Check for a null vectype.
(vect_recog_over_widening_pattern): Likewise.
(vect_recog_gather_scatter_pattern): Likewise.
(vect_recog_func_ptr): Move from tree-vectorizer.h
(vect_vect_recog_func_ptrs): Move further down the file.
(vect_recog_func): Likewise. Remove the third argument.
(NUM_PATTERNS): Define based on vect_vect_recog_func_ptrs.
(vect_pattern_recog_1): Expect the pattern function to do any
necessary target tests. Also expect it to provide a vector type.
Remove the type_in handling.
From-SVN: r261791
|
|
This message is a long write-up for a patch that simply adds a common
routine for printing the "vector_foo_pattern: detected:" messages.
The reason for doing this is that some routines check for target support
themselves and some leave it to vect_pattern_recog_1. Those that leave
it to vect_pattern_recog_1 currently print these "detected:" messages if
the statements have the right form, even if the pattern is eventually
discarded. IMO that's useful, and a lot of existing scan tests rely on it.
However, a later patch makes patterns do their own testing, and stops
them creating pattern statements until the tests have passed. This means
(a) they need to print the "detected:" message earlier and (b) the pattern
statement won't be around to print.
The patch therefore makes all routines print the original statement
rather than the pattern one. That information isn't obvious otherwise,
whereas vect_pattern_recog_1 already prints the pattern statement
in the case of a successful match. This also avoids the previous
situation in which a routine could print "detected:" and then
silently bail out before saying what had been detected.
2018-06-20 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vect-patterns.c (vect_pattern_detected): New function.
(vect_recog_dot_prod_patternm, vect_recog_sad_pattern)
(vect_recog_widen_mult_pattern, vect_recog_widen_sum_pattern)
(vect_recog_over_widening_pattern, vect_recog_widen_shift_pattern
(vect_recog_rotate_pattern, vect_recog_vector_vector_shift_pattern)
(vect_recog_mult_pattern, vect_recog_divmod_pattern)
(vect_recog_mixed_size_cond_pattern, vect_recog_bool_pattern)
(vect_recog_mask_conversion_pattern)
(vect_try_gather_scatter_pattern): Likewise.
From-SVN: r261790
|
|
This patch adds a helper for pattern code that wants to find an
internal (vectorisable) definition of an SSA name.
A later patch will make more use of this, and alter the definition.
2018-06-20 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vect-patterns.c (vect_get_internal_def): New function.
(vect_recog_dot_prod_pattern, vect_recog_sad_pattern)
(vect_recog_vector_vector_shift_pattern, check_bool_pattern)
(search_type_for_mask_1): Use it.
From-SVN: r261789
|
|
vect_recog_dot_prod_pattern and vect_recog_sad_pattern both checked
whether the statement passed in had already been recognised as a
WIDEN_SUM_EXPR pattern. That isn't possible (any more?), since the
first recognised pattern wins, and since vect_recog_widen_sum_pattern
never matches a later statement than the one it's given.
2018-06-20 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vect-patterns.c (vect_recog_dot_prod_pattern): Remove
redundant WIDEN_SUM_EXPR handling.
(vect_recog_sad_pattern): Likewise.
From-SVN: r261788
|
|
tree-vect-patterns.c checked that operands to primitive arithmetic ops
are compatible with each other and with the result. The checks date
back years and have long been redundant with verify_gimple_stmt.
2018-06-20 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vect-patterns.c (vect_recog_dot_prod_pattern): Remove
redundant check that the types of a PLUS_EXPR or MULT_EXPR agree.
(vect_recog_sad_pattern): Likewise PLUS_EXPR, ABS_EXPR and MINUS_EXPR.
(vect_recog_widen_mult_pattern): Likewise MULT_EXPR.
(vect_recog_widen_sum_pattern): Likewise PLUS_EXPR.
From-SVN: r261787
|
|
A pattern's PATTERN_DEF_SEQ was attached to both the original statement
and the main pattern statement, which made it harder to update later.
This patch attaches it to just the original statement. In practice,
anything that cared had ready access to both.
2018-06-20 Richard Sandiford <richard.sandiford@arm.com>
gcc/
* tree-vectorizer.h (_stmt_vec_info): Note above pattern_def_seq
that the sequence is attached to the original statement rather
than the pattern statement.
* tree-vect-loop.c (vect_determine_vf_for_stmt): Take the
PATTERN_DEF_SEQ from the original statement rather than
the main pattern statement.
* tree-vect-stmts.c (free_stmt_vec_info): Likewise.
* tree-vect-patterns.c (vect_recog_dot_prod_pattern): Likewise.
(vect_mark_pattern_stmts): Don't copy the PATTERN_DEF_SEQ.
From-SVN: r261785
|
|
2018-06-19 Richard Biener <rguenther@suse.de>
PR tree-optimization/86179
* tree-vect-patterns.c (vect_pattern_recog_1): Clean up
after failed recognition.
* gcc.dg/pr86179.c: New testcase.
From-SVN: r261731
|
|
gcc/ChangeLog:
* tree-vect-data-refs.c (vect_analyze_data_ref_dependences):
Replace dump_printf_loc call with DUMP_VECT_SCOPE.
(vect_slp_analyze_instance_dependence): Likewise.
(vect_enhance_data_refs_alignment): Likewise.
(vect_analyze_data_refs_alignment): Likewise.
(vect_slp_analyze_and_verify_instance_alignment
(vect_analyze_data_ref_accesses): Likewise.
(vect_prune_runtime_alias_test_list): Likewise.
(vect_analyze_data_refs): Likewise.
* tree-vect-loop-manip.c (vect_update_inits_of_drs): Likewise.
* tree-vect-loop.c (vect_determine_vectorization_factor): Likewise.
(vect_analyze_scalar_cycles_1): Likewise.
(vect_get_loop_niters): Likewise.
(vect_analyze_loop_form_1): Likewise.
(vect_update_vf_for_slp): Likewise.
(vect_analyze_loop_operations): Likewise.
(vect_analyze_loop): Likewise.
(vectorizable_induction): Likewise.
(vect_transform_loop): Likewise.
* tree-vect-patterns.c (vect_pattern_recog): Likewise.
* tree-vect-slp.c (vect_analyze_slp): Likewise.
(vect_make_slp_decision): Likewise.
(vect_detect_hybrid_slp): Likewise.
(vect_slp_analyze_operations): Likewise.
(vect_slp_bb): Likewise.
* tree-vect-stmts.c (vect_mark_stmts_to_be_vectorized): Likewise.
(vectorizable_bswap): Likewise.
(vectorizable_call): Likewise.
(vectorizable_simd_clone_call): Likewise.
(vectorizable_conversion): Likewise.
(vectorizable_assignment): Likewise.
(vectorizable_shift): Likewise.
(vectorizable_operation): Likewise.
* tree-vectorizer.h (DUMP_VECT_SCOPE): New macro.
From-SVN: r261710
|
|
gcc.target/aarch64/vect-abs-compile.c - "abs" vectorization fails for char/short types)
gcc/ChangeLog:
2018-06-16 Kugan Vivekanandarajah <kuganv@linaro.org>
PR middle-end/64946
* cfgexpand.c (expand_debug_expr): Hande ABSU_EXPR.
* config/i386/i386.c (ix86_add_stmt_cost): Likewise.
* dojump.c (do_jump): Likewise.
* expr.c (expand_expr_real_2): Check operand type's sign.
* fold-const.c (const_unop): Handle ABSU_EXPR.
(fold_abs_const): Likewise.
* gimple-pretty-print.c (dump_unary_rhs): Likewise.
* gimple-ssa-backprop.c (backprop::process_assign_use): Likesie.
(strip_sign_op_1): Likesise.
* match.pd: Add new pattern to generate ABSU_EXPR.
* optabs-tree.c (optab_for_tree_code): Handle ABSU_EXPR.
* tree-cfg.c (verify_gimple_assign_unary): Likewise.
* tree-eh.c (operation_could_trap_helper_p): Likewise.
* tree-inline.c (estimate_operator_cost): Likewise.
* tree-pretty-print.c (dump_generic_node): Likewise.
* tree-vect-patterns.c (vect_recog_sad_pattern): Likewise.
* tree.def (ABSU_EXPR): New.
gcc/c-family/ChangeLog:
2018-06-16 Kugan Vivekanandarajah <kuganv@linaro.org>
* c-common.c (c_common_truthvalue_conversion): Handle ABSU_EXPR.
gcc/c/ChangeLog:
2018-06-16 Kugan Vivekanandarajah <kuganv@linaro.org>
* c-typeck.c (build_unary_op): Handle ABSU_EXPR;
* gimple-parser.c (c_parser_gimple_statement): Likewise.
(c_parser_gimple_unary_expression): Likewise.
gcc/cp/ChangeLog:
2018-06-16 Kugan Vivekanandarajah <kuganv@linaro.org>
* constexpr.c (potential_constant_expression_1): Handle ABSU_EXPR.
* cp-gimplify.c (cp_fold): Likewise.
gcc/testsuite/ChangeLog:
2018-06-16 Kugan Vivekanandarajah <kuganv@linaro.org>
PR middle-end/64946
* gcc.dg/absu.c: New test.
* gcc.dg/gimplefe-29.c: New test.
* gcc.target/aarch64/pr64946.c: New test.
From-SVN: r261681
|
|
vector type of the intermediate stmt.
2018-06-13 Richard Biener <rguenther@suse.de>
* tree-vect-patterns.c (vect_recog_vector_vector_shift_pattern):
Properly set vector type of the intermediate stmt.
* tree-vect-stmts.c (vectorizable_operation): The destination
var always has vectype_out type.
From-SVN: r261553
|
