diff options
| author | Robin Dapp <rdapp@linux.ibm.com> | 2022-01-13 09:51:07 +0100 |
|---|---|---|
| committer | Robin Dapp <rdapp@linux.ibm.com> | 2022-01-13 20:05:18 +0100 |
| commit | b0e5163960eceab701a1a25dfa049e394fe5b3de (patch) | |
| tree | 6f06da924ca25a3ea73e937f55eda28cbcdd51f7 /gcc/tree-vect-stmts.c | |
| parent | 69561fc781aca3dea3aa4d5d562ef5a502965924 (diff) | |
| download | gcc-b0e5163960eceab701a1a25dfa049e394fe5b3de.zip gcc-b0e5163960eceab701a1a25dfa049e394fe5b3de.tar.gz gcc-b0e5163960eceab701a1a25dfa049e394fe5b3de.tar.bz2 | |
vect: Add bias parameter for partial vectorization
This introduces a bias parameter for the len_load/len_store ifns as well as
optabs that is meant to distinguish between Power and s390 variants.
PowerPC's instructions require a bias of 0, while in s390's case
vll/vstl do not support lengths of zero bytes and a bias of -1 must be used.
gcc/ChangeLog:
* internal-fn.c (expand_partial_load_optab_fn): Add bias.
(expand_partial_store_optab_fn): Likewise.
(internal_len_load_store_bias): New function.
* internal-fn.h (VECT_PARTIAL_BIAS_UNSUPPORTED): New define.
(internal_len_load_store_bias): New function.
* tree-vect-loop-manip.c (vect_set_loop_controls_directly): Set bias.
(vect_set_loop_condition_partial_vectors): Add header_seq parameter.
* tree-vect-loop.c (vect_verify_loop_lens): Verify bias.
(vect_estimate_min_profitable_iters): Account for bias.
(vect_get_loop_len): Add bias-adjusted length.
* tree-vect-stmts.c (vectorizable_store): Use.
(vectorizable_load): Use.
* tree-vectorizer.h (struct rgroup_controls): Add bias-adjusted length.
(LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS): New macro.
* config/rs6000/vsx.md: Use const0 bias predicate.
* doc/md.texi: Document bias value.
Diffstat (limited to 'gcc/tree-vect-stmts.c')
| -rw-r--r-- | gcc/tree-vect-stmts.c | 31 |
1 files changed, 22 insertions, 9 deletions
diff --git a/gcc/tree-vect-stmts.c b/gcc/tree-vect-stmts.c index f2625a2..44ef7a6 100644 --- a/gcc/tree-vect-stmts.c +++ b/gcc/tree-vect-stmts.c @@ -8416,9 +8416,15 @@ vectorizable_store (vec_info *vinfo, gsi); vec_oprnd = var; } + + signed char biasval = + LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS (loop_vinfo); + + tree bias = build_int_cst (intQI_type_node, biasval); gcall *call - = gimple_build_call_internal (IFN_LEN_STORE, 4, dataref_ptr, - ptr, final_len, vec_oprnd); + = gimple_build_call_internal (IFN_LEN_STORE, 5, dataref_ptr, + ptr, final_len, vec_oprnd, + bias); gimple_call_set_nothrow (call, true); vect_finish_stmt_generation (vinfo, stmt_info, call, gsi); new_stmt = call; @@ -9724,22 +9730,29 @@ vectorizable_load (vec_info *vinfo, vec_num * j + i); tree ptr = build_int_cst (ref_type, align * BITS_PER_UNIT); + + machine_mode vmode = TYPE_MODE (vectype); + opt_machine_mode new_ovmode + = get_len_load_store_mode (vmode, true); + machine_mode new_vmode = new_ovmode.require (); + tree qi_type = unsigned_intQI_type_node; + + signed char biasval = + LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS (loop_vinfo); + + tree bias = build_int_cst (intQI_type_node, biasval); + gcall *call - = gimple_build_call_internal (IFN_LEN_LOAD, 3, + = gimple_build_call_internal (IFN_LEN_LOAD, 4, dataref_ptr, ptr, - final_len); + final_len, bias); gimple_call_set_nothrow (call, true); new_stmt = call; data_ref = NULL_TREE; /* Need conversion if it's wrapped with VnQI. */ - machine_mode vmode = TYPE_MODE (vectype); - opt_machine_mode new_ovmode - = get_len_load_store_mode (vmode, true); - machine_mode new_vmode = new_ovmode.require (); if (vmode != new_vmode) { - tree qi_type = unsigned_intQI_type_node; tree new_vtype = build_vector_type_for_mode (qi_type, new_vmode); tree var = vect_get_new_ssa_name (new_vtype, |