diff options
| author | Richard Biener <rguenther@suse.de> | 2021-08-17 15:50:31 +0200 |
|---|---|---|
| committer | Richard Biener <rguenther@suse.de> | 2021-08-17 18:54:24 +0200 |
| commit | e7e17be3be3d43640a9a4a33d59315dec69fdda4 (patch) | |
| tree | 1200fa8df167b20f99c29bbf2a69f7dadbb63181 /gcc/tree-vect-data-refs.c | |
| parent | 817766f4dd2f6f3fdea2c4e6e22358b0b6b06d0d (diff) | |
| download | gcc-e7e17be3be3d43640a9a4a33d59315dec69fdda4.zip gcc-e7e17be3be3d43640a9a4a33d59315dec69fdda4.tar.gz gcc-e7e17be3be3d43640a9a4a33d59315dec69fdda4.tar.bz2 | |
Fall back to masked_gather_load/masked_scatter_store
This adds a fallback to the masked_ variants for gather_load
and scatter_store if the latter are not available.
2021-08-17 Richard Biener <rguenther@suse.de>
* optabs-query.c (supports_vec_gather_load_p): Also check
for masked optabs.
(supports_vec_scatter_store_p): Likewise.
* tree-vect-data-refs.c (vect_gather_scatter_fn_p): Fall
back to masked variants if non-masked are not supported.
* tree-vect-patterns.c (vect_recog_gather_scatter_pattern):
When we need to use masked gather/scatter but do not have
a mask set up a constant true one.
* tree-vect-stmts.c (vect_check_scalar_mask): Also allow
non-SSA_NAME masks.
Diffstat (limited to 'gcc/tree-vect-data-refs.c')
| -rw-r--r-- | gcc/tree-vect-data-refs.c | 22 |
1 files changed, 19 insertions, 3 deletions
diff --git a/gcc/tree-vect-data-refs.c b/gcc/tree-vect-data-refs.c index d594c0a..b7dde07 100644 --- a/gcc/tree-vect-data-refs.c +++ b/gcc/tree-vect-data-refs.c @@ -3735,11 +3735,17 @@ vect_gather_scatter_fn_p (vec_info *vinfo, bool read_p, bool masked_p, return false; /* Work out which function we need. */ - internal_fn ifn; + internal_fn ifn, alt_ifn; if (read_p) - ifn = masked_p ? IFN_MASK_GATHER_LOAD : IFN_GATHER_LOAD; + { + ifn = masked_p ? IFN_MASK_GATHER_LOAD : IFN_GATHER_LOAD; + alt_ifn = IFN_MASK_GATHER_LOAD; + } else - ifn = masked_p ? IFN_MASK_SCATTER_STORE : IFN_SCATTER_STORE; + { + ifn = masked_p ? IFN_MASK_SCATTER_STORE : IFN_SCATTER_STORE; + alt_ifn = IFN_MASK_SCATTER_STORE; + } for (;;) { @@ -3755,6 +3761,16 @@ vect_gather_scatter_fn_p (vec_info *vinfo, bool read_p, bool masked_p, *offset_vectype_out = offset_vectype; return true; } + else if (!masked_p + && internal_gather_scatter_fn_supported_p (alt_ifn, vectype, + memory_type, + offset_vectype, + scale)) + { + *ifn_out = alt_ifn; + *offset_vectype_out = offset_vectype; + return true; + } if (TYPE_PRECISION (offset_type) >= POINTER_SIZE && TYPE_PRECISION (offset_type) >= element_bits) |