diff options
| author | Richard Biener <rguenther@suse.de> | 2024-11-04 12:58:41 +0100 |
|---|---|---|
| committer | Richard Biener <rguenth@gcc.gnu.org> | 2024-11-07 13:50:49 +0100 |
| commit | 42d99f63cfccabe1d19177993abf4f1219d6f967 (patch) | |
| tree | 3c02ef2910eed5392fa08f2246676d39a7bc1c2b /gcc/tree-vectorizer.h | |
| parent | 2c25af0e52a631e46a1731594301e5e63bc28992 (diff) | |
| download | gcc-42d99f63cfccabe1d19177993abf4f1219d6f967.zip gcc-42d99f63cfccabe1d19177993abf4f1219d6f967.tar.gz gcc-42d99f63cfccabe1d19177993abf4f1219d6f967.tar.bz2 | |
Add LOOP_VINFO_MAIN_LOOP_INFO
The following introduces LOOP_VINFO_MAIN_LOOP_INFO alongside
LOOP_VINFO_ORIG_LOOP_INFO so one can have both access to the main
vectorized loop info and the preceeding vectorized epilogue.
This is critical for correctness as we need to disallow never
executed epilogues by costing in vect_analyze_loop_costing as
we assume those do not exist when deciding to add a skip-vector
edge during peeling. The patch also changes how multiple vector
epilogues are handled - instead of the epilogue_vinfos array in
the main loop info we now record the single epilogue_vinfo there
and further epilogues in the epilogue_vinfo member of the
epilogue info. This simplifies code.
* tree-vectorizer.h (_loop_vec_info::main_loop_info): New.
(LOOP_VINFO_MAIN_LOOP_INFO): Likewise.
(_loop_vec_info::epilogue_vinfo): Change from epilogue_vinfos
from array to single element.
* tree-vect-loop.cc (_loop_vec_info::_loop_vec_info): Initialize
main_loop_info and epilogue_vinfo. Remove epilogue_vinfos
allocation.
(_loop_vec_info::~_loop_vec_info): Do not release epilogue_vinfos.
(vect_create_loop_vinfo): Rename parameter, set
LOOP_VINFO_MAIN_LOOP_INFO.
(vect_analyze_loop_1): Rename parameter.
(vect_analyze_loop_costing): Properly distinguish between
the main vector loop and the preceeding epilogue.
(vect_analyze_loop): Change for epilogue_vinfos no longer
being a vector.
* tree-vect-loop-manip.cc (vect_do_peeling): Simplify and
thereby handle a vector epilogue of a vector epilogue.
Diffstat (limited to 'gcc/tree-vectorizer.h')
| -rw-r--r-- | gcc/tree-vectorizer.h | 12 |
1 files changed, 9 insertions, 3 deletions
diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h index 5a1bd23..88db34b 100644 --- a/gcc/tree-vectorizer.h +++ b/gcc/tree-vectorizer.h @@ -987,12 +987,17 @@ public: class loop *scalar_loop; /* For loops being epilogues of already vectorized loops - this points to the original vectorized loop. Otherwise NULL. */ + this points to the main vectorized loop. Otherwise NULL. */ + _loop_vec_info *main_loop_info; + + /* For loops being epilogues of already vectorized loops + this points to the preceeding vectorized (possibly epilogue) loop. + Otherwise NULL. */ _loop_vec_info *orig_loop_info; - /* Used to store loop_vec_infos of epilogues of this loop during + /* Used to store loop_vec_infos of the epilogue of this loop during analysis. */ - vec<_loop_vec_info *> epilogue_vinfos; + _loop_vec_info *epilogue_vinfo; /* If this is an epilogue loop the DR advancement applied. */ tree drs_advanced_by; @@ -1096,6 +1101,7 @@ public: #define LOOP_VINFO_SCALAR_LOOP_SCALING(L) (L)->scalar_loop_scaling #define LOOP_VINFO_HAS_MASK_STORE(L) (L)->has_mask_store #define LOOP_VINFO_SCALAR_ITERATION_COST(L) (L)->scalar_cost_vec +#define LOOP_VINFO_MAIN_LOOP_INFO(L) (L)->main_loop_info #define LOOP_VINFO_ORIG_LOOP_INFO(L) (L)->orig_loop_info #define LOOP_VINFO_SIMD_IF_COND(L) (L)->simd_if_cond #define LOOP_VINFO_INNER_LOOP_COST_FACTOR(L) (L)->inner_loop_cost_factor |