diff options
author | Richard Biener <rguenther@suse.de> | 2024-03-01 09:29:32 +0100 |
---|---|---|
committer | Richard Biener <rguenther@suse.de> | 2024-05-10 15:04:48 +0200 |
commit | 52d4691294c84793b301ad3cc24e277b8c7efe0b (patch) | |
tree | 88d7d72b2e90cc936c95f1aa749d787c34940e08 /gcc/tree-vect-patterns.cc | |
parent | af64af69c3cc85dbe00c520651a54850bf5cadc1 (diff) | |
download | gcc-52d4691294c84793b301ad3cc24e277b8c7efe0b.zip gcc-52d4691294c84793b301ad3cc24e277b8c7efe0b.tar.gz gcc-52d4691294c84793b301ad3cc24e277b8c7efe0b.tar.bz2 |
Allow patterns in SLP reductions
The following removes the over-broad rejection of patterns for SLP
reductions which is done by removing them from LOOP_VINFO_REDUCTIONS
during pattern detection. That's also insufficient in case the
pattern only appears on the reduction path. Instead this implements
the proper correctness check in vectorizable_reduction and guides
SLP discovery to heuristically avoid forming later invalid groups.
I also couldn't find any testcase that FAILs when allowing the SLP
reductions to form so I've added one.
I came across this for single-lane SLP reductions with the all-SLP
work where we rely on patterns to properly vectorize COND_EXPR
reductions.
* tree-vect-patterns.cc (vect_pattern_recog_1): Do not
remove reductions involving patterns.
* tree-vect-loop.cc (vectorizable_reduction): Reject SLP
reduction groups with multiple lane-reducing reductions.
* tree-vect-slp.cc (vect_analyze_slp_instance): When discovering
SLP reduction groups avoid including lane-reducing ones.
* gcc.dg/vect/vect-reduc-sad-9.c: New testcase.
Diffstat (limited to 'gcc/tree-vect-patterns.cc')
-rw-r--r-- | gcc/tree-vect-patterns.cc | 13 |
1 files changed, 0 insertions, 13 deletions
diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc index 8e8de5e..dfb7d80 100644 --- a/gcc/tree-vect-patterns.cc +++ b/gcc/tree-vect-patterns.cc @@ -7160,7 +7160,6 @@ vect_pattern_recog_1 (vec_info *vinfo, vect_recog_func *recog_func, stmt_vec_info stmt_info) { gimple *pattern_stmt; - loop_vec_info loop_vinfo; tree pattern_vectype; /* If this statement has already been replaced with pattern statements, @@ -7186,8 +7185,6 @@ vect_pattern_recog_1 (vec_info *vinfo, return; } - loop_vinfo = dyn_cast <loop_vec_info> (vinfo); - /* Found a vectorizable pattern. */ if (dump_enabled_p ()) dump_printf_loc (MSG_NOTE, vect_location, @@ -7196,16 +7193,6 @@ vect_pattern_recog_1 (vec_info *vinfo, /* Mark the stmts that are involved in the pattern. */ vect_mark_pattern_stmts (vinfo, stmt_info, pattern_stmt, pattern_vectype); - - /* Patterns cannot be vectorized using SLP, because they change the order of - computation. */ - if (loop_vinfo) - { - unsigned ix, ix2; - stmt_vec_info *elem_ptr; - VEC_ORDERED_REMOVE_IF (LOOP_VINFO_REDUCTIONS (loop_vinfo), ix, ix2, - elem_ptr, *elem_ptr == stmt_info); - } } |