diff options
author | liuhongt <hongtao.liu@intel.com> | 2023-02-01 13:30:12 +0800 |
---|---|---|
committer | liuhongt <hongtao.liu@intel.com> | 2023-02-02 17:02:01 +0800 |
commit | 209f02b0a9e9adc0bf0247cb5eef04e0f175d64e (patch) | |
tree | 536170ab9e00d799297b6d85f3258b604107ba77 /gcc/tree-vect-loop-manip.cc | |
parent | 0f349928e16fdc7dba52561e8d40347909f9f0ff (diff) | |
download | gcc-209f02b0a9e9adc0bf0247cb5eef04e0f175d64e.zip gcc-209f02b0a9e9adc0bf0247cb5eef04e0f175d64e.tar.gz gcc-209f02b0a9e9adc0bf0247cb5eef04e0f175d64e.tar.bz2 |
Don't peel nonlinear iv(mult or shift) for epilog when vf is not constant.
Normally when vf is not constant, it will be prevented by
vectorizable_nonlinear_inductions, but for this case, it failed going
into
if (STMT_VINFO_RELEVANT_P (stmt_info))
{
need_to_vectorize = true;
if (STMT_VINFO_DEF_TYPE (stmt_info) == vect_induction_def
&& ! PURE_SLP_STMT (stmt_info))
ok = vectorizable_induction (loop_vinfo,
stmt_info, NULL, NULL,
&cost_vec);
since the iv is never used outside of the loop, and will be dce later, so
vectorizer doesn't bother checking if it's vectorizable. it's
true but hit gcc_assert in vect_can_peel_nonlinear_iv_p when vf is not
constant. One solution is ignoring the nonlinear iv peeling if it's
!STMT_VINFO_RELEVANT_P (stmt_info) just like the upper code, the other
solution is returning false earlier in the
vect_can_peel_nonlinear_iv_p when vf is not constant, the patch chooses
the second incase there's other cases using vect_can_advance_ivs_p which
calls vect_can_peel_nonlinear_iv_p.
Also remove vect_peel_nonlinear_iv_p from
vectorizable_nonlinear_inductions.
gcc/ChangeLog:
PR tree-optimization/108601
* tree-vectorizer.h (vect_can_peel_nonlinear_iv_p): Removed.
* tree-vect-loop.cc
(vectorizable_nonlinear_induction): Remove
vect_can_peel_nonlinear_iv_p.
(vect_can_peel_nonlinear_iv_p): Don't peel
nonlinear iv(mult or shift) for epilog when vf is not
constant and moved the defination to ..
* tree-vect-loop-manip.cc (vect_can_peel_nonlinear_iv_p):
.. Here.
gcc/testsuite/ChangeLog:
* gcc.target/aarch64/pr108601.c: New test.
Diffstat (limited to 'gcc/tree-vect-loop-manip.cc')
-rw-r--r-- | gcc/tree-vect-loop-manip.cc | 44 |
1 files changed, 44 insertions, 0 deletions
diff --git a/gcc/tree-vect-loop-manip.cc b/gcc/tree-vect-loop-manip.cc index b5c5f85..c04fcf4 100644 --- a/gcc/tree-vect-loop-manip.cc +++ b/gcc/tree-vect-loop-manip.cc @@ -1390,6 +1390,50 @@ iv_phi_p (stmt_vec_info stmt_info) return true; } +/* Return true if vectorizer can peel for nonlinear iv. */ +static bool +vect_can_peel_nonlinear_iv_p (loop_vec_info loop_vinfo, + enum vect_induction_op_type induction_type) +{ + tree niters_skip; + /* Init_expr will be update by vect_update_ivs_after_vectorizer, + if niters or vf is unkown: + For shift, when shift mount >= precision, there would be UD. + For mult, don't known how to generate + init_expr * pow (step, niters) for variable niters. + For neg, it should be ok, since niters of vectorized main loop + will always be multiple of 2. */ + if ((!LOOP_VINFO_NITERS_KNOWN_P (loop_vinfo) + || !LOOP_VINFO_VECT_FACTOR (loop_vinfo).is_constant ()) + && induction_type != vect_step_op_neg) + { + if (dump_enabled_p ()) + dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, + "Peeling for epilogue is not supported" + " for nonlinear induction except neg" + " when iteration count is unknown.\n"); + return false; + } + + /* Also doens't support peel for neg when niter is variable. + ??? generate something like niter_expr & 1 ? init_expr : -init_expr? */ + niters_skip = LOOP_VINFO_MASK_SKIP_NITERS (loop_vinfo); + if ((niters_skip != NULL_TREE + && TREE_CODE (niters_skip) != INTEGER_CST) + || (!vect_use_loop_mask_for_alignment_p (loop_vinfo) + && LOOP_VINFO_PEELING_FOR_ALIGNMENT (loop_vinfo) < 0)) + { + if (dump_enabled_p ()) + dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, + "Peeling for alignement is not supported" + " for nonlinear induction when niters_skip" + " is not constant.\n"); + return false; + } + + return true; +} + /* Function vect_can_advance_ivs_p In case the number of iterations that LOOP iterates is unknown at compile |