diff options
| author | Richard Biener <rguenther@suse.de> | 2021-11-02 18:47:14 +0100 |
|---|---|---|
| committer | Richard Biener <rguenther@suse.de> | 2021-11-02 18:49:42 +0100 |
| commit | 164bbf701ff10ff44e272525e8f462ed3ff1cf43 (patch) | |
| tree | 44863a3d09e9e8f5ea68165e9088a12a125d49e4 /gcc/tree-vect-loop-manip.c | |
| parent | f19791565d7ed31c2a895fc29b772c2c7ed390e8 (diff) | |
| download | gcc-164bbf701ff10ff44e272525e8f462ed3ff1cf43.zip gcc-164bbf701ff10ff44e272525e8f462ed3ff1cf43.tar.gz gcc-164bbf701ff10ff44e272525e8f462ed3ff1cf43.tar.bz2 | |
tree-optimization/103029 - ensure vect loop versioning constraint on PHIs
PHI nodes in vectorizer loop versioning need to maintain the same
order of PHI arguments to not disturb SLP discovery. The following
adds an assertion and mitigation in case loop versioning breaks this
which happens more often after the recent reorg.
2021-11-02 Richard Biener <rguenther@suse.de>
PR tree-optimization/103029
* tree-vect-loop-manip.c (vect_loop_versioning): Ensure
the PHI nodes in the loop maintain their original operand
order.
Diffstat (limited to 'gcc/tree-vect-loop-manip.c')
| -rw-r--r-- | gcc/tree-vect-loop-manip.c | 17 |
1 files changed, 17 insertions, 0 deletions
diff --git a/gcc/tree-vect-loop-manip.c b/gcc/tree-vect-loop-manip.c index 72d5831..f788deb 100644 --- a/gcc/tree-vect-loop-manip.c +++ b/gcc/tree-vect-loop-manip.c @@ -3562,12 +3562,29 @@ vect_loop_versioning (loop_vec_info loop_vinfo, "applying loop versioning to outer loop %d\n", loop_to_version->num); + unsigned orig_pe_idx = loop_preheader_edge (loop)->dest_idx; + initialize_original_copy_tables (); nloop = loop_version (loop_to_version, cond_expr, &condition_bb, prob, prob.invert (), prob, prob.invert (), true); gcc_assert (nloop); nloop = get_loop_copy (loop); + /* For cycle vectorization with SLP we rely on the PHI arguments + appearing in the same order as the SLP node operands which for the + loop PHI nodes means the preheader edge dest index needs to remain + the same for the analyzed loop which also becomes the vectorized one. + Make it so in case the state after versioning differs by redirecting + the first edge into the header to the same destination which moves + it last. */ + if (loop_preheader_edge (loop)->dest_idx != orig_pe_idx) + { + edge e = EDGE_PRED (loop->header, 0); + ssa_redirect_edge (e, e->dest); + flush_pending_stmts (e); + } + gcc_assert (loop_preheader_edge (loop)->dest_idx == orig_pe_idx); + /* Kill off IFN_LOOP_VECTORIZED_CALL in the copy, nobody will reap those otherwise; they also refer to the original loops. */ |