vect: Extend peeling and versioning for alignment to VLA modes

This patch extends the support for peeling and versioning for alignment
from VLS modes to VLA modes. The key change is allowing the DR target
alignment to be set to a non-constant poly_int. Since the value must be
a power-of-two, for variable VFs, the power-of-two check is deferred to
runtime through loop versioning. The vectorizable check for speculative
loads is also refactored in this patch to handle both constant and
variable target alignment values.

Additional changes for VLA modes include:

1) Peeling

In VLA modes, we use peeling with masking - using a partial vector in
the first iteration of the vectorized loop to ensure aligned DRs in
subsequent iterations. It was already enabled for VLS modes to avoid
scalar peeling. This patch reuses most of the existing logic and just
fixes a small issue of incorrect IV offset in VLA code path. This also
removes a power-of-two rounding when computing the number of iterations
to peel, as power-of-two VF has been guaranteed by a new runtime check.

2) Versioning

The type of the mask for runtime alignment check is updated to poly_int
to support variable VFs. After this change, both standalone versioning
and peeling with versioning are available in VLA modes. This patch also
introduces another runtime check for speculative read amount, to ensure
that all speculative loads remain within current valid memory page. We
plan to remove these runtime checks in the future by introducing capped
VF - using partial vectors to limit the actual VF value at runtime.

3) Speculative read flag

DRs whose scalar accesses are known to be in-bounds will be considered
unaligned unsupported with a variable target alignment. But in fact,
speculative reads can be naturally avoided for in-bounds DRs as long as
partial vectors are used. Therefore, this patch clears the speculative
flags and sets the "must use partial vectors" flag for these cases.

This patch is bootstrapped and regression-tested on x86_64-linux-gnu,
arm-linux-gnueabihf and aarch64-linux-gnu with bootstrap-O3.

gcc/ChangeLog:

	* tree-vect-data-refs.cc (vect_compute_data_ref_alignment):
	Allow DR target alignment to be a poly_int.
	(vect_enhance_data_refs_alignment): Support peeling and
	versioning for VLA modes.
	* tree-vect-loop-manip.cc (get_misalign_in_elems): Remove
	power-of-two rounding in peeling.
	(vect_create_cond_for_align_checks): Update alignment check
	logic for poly_int mask.
	(vect_create_cond_for_vla_spec_read): New runtime checks.
	(vect_loop_versioning): Support new runtime checks.
	* tree-vect-loop.cc (_loop_vec_info::_loop_vec_info): Add a new
	loop_vinfo field.
	(vectorizable_induction): Fix wrong IV offset issue.
	* tree-vect-stmts.cc (get_load_store_type): Refactor
	vectorizable checks for speculative loads.
	* tree-vectorizer.h (LOOP_VINFO_MAX_SPEC_READ_AMOUNT): New
	macro for new runtime checks.
	(LOOP_REQUIRES_VERSIONING_FOR_SPEC_READ): Likewise
	(LOOP_REQUIRES_VERSIONING): Update macro for new runtime checks.

gcc/testsuite/ChangeLog:

	* gcc.target/aarch64/sve/peel_ind_11.c: New test.
	* gcc.target/aarch64/sve/peel_ind_11_run.c: New test.
	* gcc.target/aarch64/sve/peel_ind_12.c: New test.
	* gcc.target/aarch64/sve/peel_ind_12_run.c: New test.
	* gcc.target/aarch64/sve/peel_ind_13.c: New test.
	* gcc.target/aarch64/sve/peel_ind_13_run.c: New test.

0 files changed, 0 insertions, 0 deletions