aarch64: Switch PSTATE.SM around calls

This patch adds support for switching to the appropriate SME mode
for each call.  Switching to streaming mode requires an SMSTART SM
instruction and switching to non-streaming mode requires an SMSTOP SM
instruction.  If the call is being made from streaming-compatible code,
these switches are conditional on the current mode being the opposite
of the one that the call needs.

Since changing PSTATE.SM changes the vector length and effectively
changes the ISA, the code to do the switching has to be emitted late.
The patch does this using a new pass that runs next to late prologue/
epilogue insertion.  (It doesn't use md_reorg because later additions
need the CFG.)

If a streaming-compatible function needs to switch mode for a call,
it must restore the original mode afterwards.  The old mode must
therefore be available immediately after the call.  The easiest
way of ensuring this is to force the use of a hard frame pointer
and ensure that the old state is saved at an in-range offset
from there.

Changing modes clobbers the Z and P registers, so we need to
save and restore live Z and P state around each mode switch.
However, mode switches are not expected to be performance
critical, so it seemed better to err on the side of being
correct rather than trying to optimise the save and restore
with surrounding code.

gcc/
	* config/aarch64/aarch64-passes.def
	(pass_late_thread_prologue_and_epilogue): New pass.
	* config/aarch64/aarch64-sme.md: New file.
	* config/aarch64/aarch64.md: Include it.
	(*tb<optab><mode>1): Rename to...
	(@aarch64_tb<optab><mode>): ...this.
	(call, call_value, sibcall, sibcall_value): Don't require operand 2
	to be a CONST_INT.
	* config/aarch64/aarch64-protos.h (aarch64_emit_call_insn): Return
	the insn.
	(make_pass_switch_sm_state): Declare.
	* config/aarch64/aarch64.h (TARGET_STREAMING_COMPATIBLE): New macro.
	(CALL_USED_REGISTER): Mark VG as call-preserved.
	(aarch64_frame::old_svcr_offset): New member variable.
	(machine_function::call_switches_sm_state): Likewise.
	(CUMULATIVE_ARGS::num_sme_mode_switch_args): Likewise.
	(CUMULATIVE_ARGS::sme_mode_switch_args): Likewise.
	* config/aarch64/aarch64.cc: Include tree-pass.h and cfgbuild.h.
	(aarch64_cfun_incoming_pstate_sm): New function.
	(aarch64_call_switches_pstate_sm): Likewise.
	(aarch64_reg_save_mode): Return DImode for VG_REGNUM.
	(aarch64_callee_isa_mode): New function.
	(aarch64_insn_callee_isa_mode): Likewise.
	(aarch64_guard_switch_pstate_sm): Likewise.
	(aarch64_switch_pstate_sm): Likewise.
	(aarch64_sme_mode_switch_regs): New class.
	(aarch64_record_sme_mode_switch_args): New function.
	(aarch64_finish_sme_mode_switch_args): Likewise.
	(aarch64_function_arg): Handle the end marker by returning a
	PARALLEL that contains the ABI cookie that we used previously
	alongside the result of aarch64_finish_sme_mode_switch_args.
	(aarch64_init_cumulative_args): Initialize num_sme_mode_switch_args.
	(aarch64_function_arg_advance): If a call would switch SM state,
	record all argument registers that would need to be saved around
	the mode switch.
	(aarch64_need_old_pstate_sm): New function.
	(aarch64_layout_frame): Decide whether the frame needs to store the
	incoming value of PSTATE.SM and allocate a save slot for it if so.
	If a function switches SME state, arrange to save the old value
	of the DWARF VG register.  Handle the case where this is the only
	register save slot above the FP.
	(aarch64_save_callee_saves): Handles saves of the DWARF VG register.
	(aarch64_get_separate_components): Prevent such saves from being
	shrink-wrapped.
	(aarch64_old_svcr_mem): New function.
	(aarch64_read_old_svcr): Likewise.
	(aarch64_guard_switch_pstate_sm): Likewise.
	(aarch64_expand_prologue): Handle saves of the DWARF VG register.
	Initialize any SVCR save slot.
	(aarch64_expand_call): Allow the cookie to be PARALLEL that contains
	both the UNSPEC_CALLEE_ABI value and a list of registers that need
	to be preserved across a change to PSTATE.SM.  If the call does
	involve such a change to PSTATE.SM, record the registers that
	would be clobbered by this process.  Also emit an instruction
	to mark the temporary change in VG.  Update call_switches_pstate_sm.
	(aarch64_emit_call_insn): Return the emitted instruction.
	(aarch64_frame_pointer_required): New function.
	(aarch64_conditional_register_usage): Prevent VG_REGNUM from being
	treated as a register operand.
	(aarch64_switch_pstate_sm_for_call): New function.
	(pass_data_switch_pstate_sm): New pass variable.
	(pass_switch_pstate_sm): New pass class.
	(make_pass_switch_pstate_sm): New function.
	(TARGET_FRAME_POINTER_REQUIRED): Define.
	* config/aarch64/t-aarch64 (s-check-sve-md): Add aarch64-sme.md.

gcc/testsuite/
	* gcc.target/aarch64/sme/call_sm_switch_1.c: New test.
	* gcc.target/aarch64/sme/call_sm_switch_2.c: Likewise.
	* gcc.target/aarch64/sme/call_sm_switch_3.c: Likewise.
	* gcc.target/aarch64/sme/call_sm_switch_4.c: Likewise.
	* gcc.target/aarch64/sme/call_sm_switch_5.c: Likewise.
	* gcc.target/aarch64/sme/call_sm_switch_6.c: Likewise.
	* gcc.target/aarch64/sme/call_sm_switch_7.c: Likewise.
	* gcc.target/aarch64/sme/call_sm_switch_8.c: Likewise.
	* gcc.target/aarch64/sme/call_sm_switch_9.c: Likewise.
	* gcc.target/aarch64/sme/call_sm_switch_10.c: Likewise.

0 files changed, 0 insertions, 0 deletions