diff options
| author | Richard Biener <rguenther@suse.de> | 2025-05-25 19:28:54 +0200 |
|---|---|---|
| committer | Richard Biener <rguenth@gcc.gnu.org> | 2025-07-08 10:12:29 +0200 |
| commit | df64d099faf843d90e8fe29aec17d84277986ee9 (patch) | |
| tree | 79e35f6eeeab570836bb9227ad05af7e45547d1c /gcc/tree-vectorizer.h | |
| parent | d1f05661fa6c8a6ea6f59ad365a84469100e425e (diff) | |
| download | gcc-df64d099faf843d90e8fe29aec17d84277986ee9.zip gcc-df64d099faf843d90e8fe29aec17d84277986ee9.tar.gz gcc-df64d099faf843d90e8fe29aec17d84277986ee9.tar.bz2 | |
Allow the target to request a masked vector epilogue
Targets recently got the ability to request the vector mode to be
used for a vector epilogue (or the epilogue of a vector epilogue). The
following adds the ability for it to indicate the epilogue should use
loop masking, irrespective of the --param vect-partial-vector-usage
default setting.
The patch below uses a separate flag from the epilogue mode, not
addressing the issue that on x86 the vector_modes mode iteration
hook would not allow for both masked and unmasked variants to be
tried and costed given this doesn't naturally map to modes on
that target. That's left for a future exercise - turning on
cost comparison for the x86 backend would be a prerequesite there.
* tree-vectorizer.h (vector_costs::suggested_epilogue_mode):
Add masked output parameter and return m_masked_epilogue.
(vector_costs::m_masked_epilogue): New tristate flag.
(vector_costs::vector_costs): Initialize m_masked_epilogue.
* tree-vect-loop.cc (vect_analyze_loop_1): Pass in masked
flag to optionally initialize can_use_partial_vectors_p.
(vect_analyze_loop): For epilogues also get whether to use
a masked epilogue for this loop from the target and use
that for the first epilogue mode we try.
Diffstat (limited to 'gcc/tree-vectorizer.h')
| -rw-r--r-- | gcc/tree-vectorizer.h | 13 |
1 files changed, 10 insertions, 3 deletions
diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h index 66a2964..ba06c5d 100644 --- a/gcc/tree-vectorizer.h +++ b/gcc/tree-vectorizer.h @@ -1714,7 +1714,7 @@ public: unsigned int outside_cost () const; unsigned int total_cost () const; unsigned int suggested_unroll_factor () const; - machine_mode suggested_epilogue_mode () const; + machine_mode suggested_epilogue_mode (int &masked) const; protected: unsigned int record_stmt_cost (stmt_vec_info, vect_cost_model_location, @@ -1738,8 +1738,13 @@ protected: unsigned int m_suggested_unroll_factor; /* The suggested mode to be used for a vectorized epilogue or VOIDmode, - determined at finish_cost. */ + determined at finish_cost. m_masked_epilogue specifies whether the + epilogue should use masked vectorization, regardless of the + --param vect-partial-vector-usage default. If -1 then the + --param setting takes precedence. If the user explicitly specified + --param vect-partial-vector-usage then that takes precedence. */ machine_mode m_suggested_epilogue_mode; + int m_masked_epilogue; /* True if finish_cost has been called. */ bool m_finished; @@ -1755,6 +1760,7 @@ vector_costs::vector_costs (vec_info *vinfo, bool costing_for_scalar) m_costs (), m_suggested_unroll_factor(1), m_suggested_epilogue_mode(VOIDmode), + m_masked_epilogue (-1), m_finished (false) { } @@ -1815,9 +1821,10 @@ vector_costs::suggested_unroll_factor () const /* Return the suggested epilogue mode. */ inline machine_mode -vector_costs::suggested_epilogue_mode () const +vector_costs::suggested_epilogue_mode (int &masked_p) const { gcc_checking_assert (m_finished); + masked_p = m_masked_epilogue; return m_suggested_epilogue_mode; } |