diff options
author | H.J. Lu <hjl@gcc.gnu.org> | 2008-07-30 12:20:43 -0700 |
---|---|---|
committer | H.J. Lu <hjl@gcc.gnu.org> | 2008-07-30 12:20:43 -0700 |
commit | 2e3f842fe69ce4e74cfe18493429b807af271083 (patch) | |
tree | 9703441244044dbd6ab33443385887297f540e4f /gcc | |
parent | 22129589211764a55ee21f9202ab80b3772817b6 (diff) | |
download | gcc-2e3f842fe69ce4e74cfe18493429b807af271083.zip gcc-2e3f842fe69ce4e74cfe18493429b807af271083.tar.gz gcc-2e3f842fe69ce4e74cfe18493429b807af271083.tar.bz2 |
[multiple changes]
2008-07-30 Joey Ye <joey.ye@intel.com>
H.J. Lu <hongjiu.lu@intel.com>
* builtins.c (expand_builtin_setjmp_receiver): Replace
virtual_incoming_args_rtx with crtl->args.internal_arg_pointer.
(expand_builtin_apply_args_1): Likewise.
(expand_builtin_longjmp): Need DRAP for stack alignment.
(expand_builtin_apply): Likewise.
* caller-save.c (setup_save_areas): Call assign_stack_local_1
instead of assign_stack_local to allow alignment reduction.
* calls.c (emit_call_1): Need DRAP for stack alignment if
return pops.
(expand_call): Replace virtual_incoming_args_rtx with
crtl->args.internal_arg_pointer.
* stmt.c (expand_nl_goto_receiver): Likewise.
* cfgexpand.c (get_decl_align_unit): Estimate stack variable
alignment and store to stack_alignment_estimated and
max_used_stack_slot_alignment.
(expand_one_var): Likewise.
(expand_stack_alignment): New function.
(tree_expand_cfg): Initialize max_used_stack_slot_alignment
and stack_alignment_estimated fields in rtl_data. Call
expand_stack_alignment at end.
* defaults.h (INCOMING_STACK_BOUNDARY): New.
(MAX_STACK_ALIGNMENT): Likewise.
(MAX_SUPPORTED_STACK_ALIGNMENT): Likewise.
(SUPPORTS_STACK_ALIGNMENT): Likewise.
* emit-rtl.c (gen_reg_rtx): Estimate stack alignment for
stack alignment when generating virtual registers.
* function.c (assign_stack_local): Renamed to ...
(assign_stack_local_1): This. Add a parameter to indicate
if it is OK to reduce alignment.
(assign_stack_local): Use it.
(instantiate_new_reg): Instantiate virtual incoming args rtx
to vDRAP if stack realignment and DRAP is needed.
(assign_parms): Collect parameter/return type alignment and
contribute to stack_alignment_estimated.
(locate_and_pad_parm): Likewise.
(get_arg_pointer_save_area): Replace virtual_incoming_args_rtx
with crtl->args.internal_arg_pointer.
* function.h (rtl_data): Add new field drap_reg,
max_used_stack_slot_alignment, stack_alignment_estimated,
stack_realign_needed, need_drap, stack_realign_processed and
stack_realign_finalized.
(stack_realign_fp): New macro.
(stack_realign_drap): Likewise.
* global.c (compute_regsets): Frame pointer is needed when
stack is realigned. Can eliminate frame pointer when stack is
realigned and dynamic realigned argument pointer isn't used.
* reload1.c (update_eliminables): Frame pointer is needed
when stack is realigned.
(init_elim_table): Can eliminate frame pointer when stack is
realigned and dynamic realigned argument pointer isn't used.
* rtl.h (assign_stack_local_1): Declare new funtion.
* target-def.h (TARGET_UPDATE_STACK_BOUNDARY): New.
(TARGET_GET_DRAP_RTX): Likewise.
(TARGET_CALLS): Add TARGET_UPDATE_STACK_BOUNDARY and
TARGET_GET_DRAP_RTX.
* target.h (gcc_target): Add update_stack_boundary and
get_drap_rtx.
* tree-vectorizer.c (vect_can_force_dr_alignment_p): Replace
STACK_BOUNDARY with MAX_STACK_ALIGNMENT.
2008-07-30 Xuepeng Guo <xuepeng.guo@intel.com>
H.J. Lu <hongjiu.lu@intel.com>
* dwarf2out.c (dw_fde_struct): Add stack_realignment, drap_reg,
vdrap_reg, stack_realign and drap_reg_saved.
(add_cfi): Don't allow redefining CFA when DRAP is used.
(reg_save): Handle stack alignment.
(dwarf2out_frame_debug_expr): Add rules 16-20 to handle stack
alignment. Don't generate DWARF information for (set fp sp)
when DRAP is used.
(dwarf2out_begin_prologue): Initialize drap_reg and vdrap_reg
to INVALID_REGNUM.
(int_loc_descriptor): Move prototype forward. Also define if
DWARF2_UNWIND_INFO is true.
(output_cfa_loc): Handle DW_CFA_expression.
(build_cfa_aligned_loc): New.
(based_loc_descr): Update assert for stack realign. For local
variables, use sp+offset when stack is aligned without drap and
fp+offset when stack is aligned with drap. For arguments, use
cfa+offset when drap is used to align stack.
2008-07-30 Joey Ye <joey.ye@intel.com>
H.J. Lu <hongjiu.lu@intel.com>
* config/i386/i386.c (ix86_force_align_arg_pointer_string):
Break long line.
(ix86_gen_andsp): New.
(ix86_user_incoming_stack_boundary): Likewise.
(ix86_default_incoming_stack_boundary): Likewise.
(ix86_incoming_stack_boundary): Likewise.
(ix86_can_eliminate): Likewise.
(find_drap_reg): Likewise.
(ix86_update_stack_boundary): Likewise.
(ix86_get_drap_rtx): Likewise.
(ix86_finalize_stack_realign_flags): Likewise.
(TARGET_UPDATE_STACK_BOUNDARY): Likewise.
(TARGET_GET_DRAP_RTX): Likewise.
(override_options): Overide option value for new options.
(ix86_function_ok_for_sibcall): Remove check for
force_align_arg_pointer.
(ix86_handle_cconv_attribute): Likewise.
(ix86_function_regparm): Likewise.
(setup_incoming_varargs_64): Don't set stack_alignment_needed
here.
(ix86_va_start): Replace virtual_incoming_args_rtx with
crtl->args.internal_arg_pointer.
(ix86_select_alt_pic_regnum): Check DRAP register.
(ix86_save_reg): Replace force_align_arg_pointer with drap_reg.
(ix86_compute_frame_layout): Compute frame layout wrt stack
realignment.
(ix86_internal_arg_pointer): Just return
virtual_incoming_args_rtx.
(ix86_expand_prologue): Decide if stack realignment is needed
and generate prologue code accordingly.
(ix86_expand_epilogue): Generate epilogue code wrt stack
realignment is really needed or not.
* config/i386/i386.h (MAIN_STACK_BOUNDARY): New.
(ABI_STACK_BOUNDARY): Likewise.
(PREFERRED_STACK_BOUNDARY_DEFAULT): Likewise.
(STACK_REALIGN_DEFAULT): Likewise.
(INCOMING_STACK_BOUNDARY): Likewise.
(MAX_STACK_ALIGNMENT): Likewise.
(ix86_incoming_stack_boundary): Likewise.
(FORCE_PREFERRED_STACK_BOUNDARY_IN_MAIN): Removed.
(REAL_PIC_OFFSET_TABLE_REGNUM): Updated to use BX_REG.
(CAN_ELIMINATE): Defined with ix86_can_eliminate.
(machine_function): Remove force_align_arg_pointer.
* config/i386/i386.md (BX_REG): New.
(R13_REG): Likewise.
* config/i386/i386.opt (mforce_drap): New.
(mincoming-stack-boundary): Likewise.
(mstackrealign): Add Init(-1).
* config/i386/i386-protos.h (ix86_can_eliminate): New
2008-07-30 H.J. Lu <hongjiu.lu@intel.com>
* doc/extend.texi: Update force_align_arg_pointer.
* doc/invoke.texi: Document -mincoming-stack-boundary. Update
-mstackrealign.
* doc/tm.texi (MAX_STACK_ALIGNMENT): Add macro.
(INCOMING_STACK_BOUNDARY): Likewise.
(TARGET_UPDATE_STACK_BOUNDARY): New target hook.
(TARGET_GET_DRAP_RTX): Likewise.
From-SVN: r138335
Diffstat (limited to 'gcc')
-rw-r--r-- | gcc/ChangeLog | 166 | ||||
-rw-r--r-- | gcc/builtins.c | 17 | ||||
-rw-r--r-- | gcc/caller-save.c | 12 | ||||
-rw-r--r-- | gcc/calls.c | 6 | ||||
-rw-r--r-- | gcc/cfgexpand.c | 109 | ||||
-rw-r--r-- | gcc/config/i386/i386-protos.h | 1 | ||||
-rw-r--r-- | gcc/config/i386/i386.c | 480 | ||||
-rw-r--r-- | gcc/config/i386/i386.h | 42 | ||||
-rw-r--r-- | gcc/config/i386/i386.md | 2 | ||||
-rw-r--r-- | gcc/config/i386/i386.opt | 10 | ||||
-rw-r--r-- | gcc/defaults.h | 21 | ||||
-rw-r--r-- | gcc/doc/extend.texi | 12 | ||||
-rw-r--r-- | gcc/doc/invoke.texi | 26 | ||||
-rw-r--r-- | gcc/doc/tm.texi | 29 | ||||
-rw-r--r-- | gcc/dwarf2out.c | 278 | ||||
-rw-r--r-- | gcc/emit-rtl.c | 9 | ||||
-rw-r--r-- | gcc/function.c | 140 | ||||
-rw-r--r-- | gcc/function.h | 48 | ||||
-rw-r--r-- | gcc/global.c | 6 | ||||
-rw-r--r-- | gcc/reload1.c | 12 | ||||
-rw-r--r-- | gcc/rtl.h | 1 | ||||
-rw-r--r-- | gcc/stmt.c | 2 | ||||
-rw-r--r-- | gcc/target-def.h | 4 | ||||
-rw-r--r-- | gcc/target.h | 7 | ||||
-rw-r--r-- | gcc/tree-vectorizer.c | 4 |
25 files changed, 1239 insertions, 205 deletions
diff --git a/gcc/ChangeLog b/gcc/ChangeLog index 02485f7..bc455ab 100644 --- a/gcc/ChangeLog +++ b/gcc/ChangeLog @@ -1,3 +1,169 @@ +2008-07-30 Joey Ye <joey.ye@intel.com> + H.J. Lu <hongjiu.lu@intel.com> + + * builtins.c (expand_builtin_setjmp_receiver): Replace + virtual_incoming_args_rtx with crtl->args.internal_arg_pointer. + (expand_builtin_apply_args_1): Likewise. + (expand_builtin_longjmp): Need DRAP for stack alignment. + (expand_builtin_apply): Likewise. + + * caller-save.c (setup_save_areas): Call assign_stack_local_1 + instead of assign_stack_local to allow alignment reduction. + + * calls.c (emit_call_1): Need DRAP for stack alignment if + return pops. + (expand_call): Replace virtual_incoming_args_rtx with + crtl->args.internal_arg_pointer. + * stmt.c (expand_nl_goto_receiver): Likewise. + + * cfgexpand.c (get_decl_align_unit): Estimate stack variable + alignment and store to stack_alignment_estimated and + max_used_stack_slot_alignment. + (expand_one_var): Likewise. + (expand_stack_alignment): New function. + (tree_expand_cfg): Initialize max_used_stack_slot_alignment + and stack_alignment_estimated fields in rtl_data. Call + expand_stack_alignment at end. + + * defaults.h (INCOMING_STACK_BOUNDARY): New. + (MAX_STACK_ALIGNMENT): Likewise. + (MAX_SUPPORTED_STACK_ALIGNMENT): Likewise. + (SUPPORTS_STACK_ALIGNMENT): Likewise. + + * emit-rtl.c (gen_reg_rtx): Estimate stack alignment for + stack alignment when generating virtual registers. + + * function.c (assign_stack_local): Renamed to ... + (assign_stack_local_1): This. Add a parameter to indicate + if it is OK to reduce alignment. + (assign_stack_local): Use it. + (instantiate_new_reg): Instantiate virtual incoming args rtx + to vDRAP if stack realignment and DRAP is needed. + (assign_parms): Collect parameter/return type alignment and + contribute to stack_alignment_estimated. + (locate_and_pad_parm): Likewise. + (get_arg_pointer_save_area): Replace virtual_incoming_args_rtx + with crtl->args.internal_arg_pointer. + + * function.h (rtl_data): Add new field drap_reg, + max_used_stack_slot_alignment, stack_alignment_estimated, + stack_realign_needed, need_drap, stack_realign_processed and + stack_realign_finalized. + (stack_realign_fp): New macro. + (stack_realign_drap): Likewise. + + * global.c (compute_regsets): Frame pointer is needed when + stack is realigned. Can eliminate frame pointer when stack is + realigned and dynamic realigned argument pointer isn't used. + + * reload1.c (update_eliminables): Frame pointer is needed + when stack is realigned. + (init_elim_table): Can eliminate frame pointer when stack is + realigned and dynamic realigned argument pointer isn't used. + + * rtl.h (assign_stack_local_1): Declare new funtion. + + * target-def.h (TARGET_UPDATE_STACK_BOUNDARY): New. + (TARGET_GET_DRAP_RTX): Likewise. + (TARGET_CALLS): Add TARGET_UPDATE_STACK_BOUNDARY and + TARGET_GET_DRAP_RTX. + + * target.h (gcc_target): Add update_stack_boundary and + get_drap_rtx. + + * tree-vectorizer.c (vect_can_force_dr_alignment_p): Replace + STACK_BOUNDARY with MAX_STACK_ALIGNMENT. + +2008-07-30 Xuepeng Guo <xuepeng.guo@intel.com> + H.J. Lu <hongjiu.lu@intel.com> + + * dwarf2out.c (dw_fde_struct): Add stack_realignment, drap_reg, + vdrap_reg, stack_realign and drap_reg_saved. + (add_cfi): Don't allow redefining CFA when DRAP is used. + (reg_save): Handle stack alignment. + (dwarf2out_frame_debug_expr): Add rules 16-20 to handle stack + alignment. Don't generate DWARF information for (set fp sp) + when DRAP is used. + (dwarf2out_begin_prologue): Initialize drap_reg and vdrap_reg + to INVALID_REGNUM. + (int_loc_descriptor): Move prototype forward. Also define if + DWARF2_UNWIND_INFO is true. + (output_cfa_loc): Handle DW_CFA_expression. + (build_cfa_aligned_loc): New. + (based_loc_descr): Update assert for stack realign. For local + variables, use sp+offset when stack is aligned without drap and + fp+offset when stack is aligned with drap. For arguments, use + cfa+offset when drap is used to align stack. + +2008-07-30 Joey Ye <joey.ye@intel.com> + H.J. Lu <hongjiu.lu@intel.com> + + * config/i386/i386.c (ix86_force_align_arg_pointer_string): + Break long line. + (ix86_gen_andsp): New. + (ix86_user_incoming_stack_boundary): Likewise. + (ix86_default_incoming_stack_boundary): Likewise. + (ix86_incoming_stack_boundary): Likewise. + (ix86_can_eliminate): Likewise. + (find_drap_reg): Likewise. + (ix86_update_stack_boundary): Likewise. + (ix86_get_drap_rtx): Likewise. + (ix86_finalize_stack_realign_flags): Likewise. + (TARGET_UPDATE_STACK_BOUNDARY): Likewise. + (TARGET_GET_DRAP_RTX): Likewise. + (override_options): Overide option value for new options. + (ix86_function_ok_for_sibcall): Remove check for + force_align_arg_pointer. + (ix86_handle_cconv_attribute): Likewise. + (ix86_function_regparm): Likewise. + (setup_incoming_varargs_64): Don't set stack_alignment_needed + here. + (ix86_va_start): Replace virtual_incoming_args_rtx with + crtl->args.internal_arg_pointer. + (ix86_select_alt_pic_regnum): Check DRAP register. + (ix86_save_reg): Replace force_align_arg_pointer with drap_reg. + (ix86_compute_frame_layout): Compute frame layout wrt stack + realignment. + (ix86_internal_arg_pointer): Just return + virtual_incoming_args_rtx. + (ix86_expand_prologue): Decide if stack realignment is needed + and generate prologue code accordingly. + (ix86_expand_epilogue): Generate epilogue code wrt stack + realignment is really needed or not. + + * config/i386/i386.h (MAIN_STACK_BOUNDARY): New. + (ABI_STACK_BOUNDARY): Likewise. + (PREFERRED_STACK_BOUNDARY_DEFAULT): Likewise. + (STACK_REALIGN_DEFAULT): Likewise. + (INCOMING_STACK_BOUNDARY): Likewise. + (MAX_STACK_ALIGNMENT): Likewise. + (ix86_incoming_stack_boundary): Likewise. + (FORCE_PREFERRED_STACK_BOUNDARY_IN_MAIN): Removed. + (REAL_PIC_OFFSET_TABLE_REGNUM): Updated to use BX_REG. + (CAN_ELIMINATE): Defined with ix86_can_eliminate. + (machine_function): Remove force_align_arg_pointer. + + * config/i386/i386.md (BX_REG): New. + (R13_REG): Likewise. + + * config/i386/i386.opt (mforce_drap): New. + (mincoming-stack-boundary): Likewise. + (mstackrealign): Add Init(-1). + + * config/i386/i386-protos.h (ix86_can_eliminate): New + +2008-07-30 H.J. Lu <hongjiu.lu@intel.com> + + * doc/extend.texi: Update force_align_arg_pointer. + + * doc/invoke.texi: Document -mincoming-stack-boundary. Update + -mstackrealign. + + * doc/tm.texi (MAX_STACK_ALIGNMENT): Add macro. + (INCOMING_STACK_BOUNDARY): Likewise. + (TARGET_UPDATE_STACK_BOUNDARY): New target hook. + (TARGET_GET_DRAP_RTX): Likewise. + 2008-07-30 Andreas Schwab <schwab@suse.de> PR rtl-optimization/36929 diff --git a/gcc/builtins.c b/gcc/builtins.c index 288ad59..9a9c6a3 100644 --- a/gcc/builtins.c +++ b/gcc/builtins.c @@ -743,7 +743,7 @@ expand_builtin_setjmp_receiver (rtx receiver_label ATTRIBUTE_UNUSED) { /* Now restore our arg pointer from the address at which it was saved in our stack frame. */ - emit_move_insn (virtual_incoming_args_rtx, + emit_move_insn (crtl->args.internal_arg_pointer, copy_to_reg (get_arg_pointer_save_area ())); } } @@ -778,6 +778,11 @@ expand_builtin_longjmp (rtx buf_addr, rtx value) rtx fp, lab, stack, insn, last; enum machine_mode sa_mode = STACK_SAVEAREA_MODE (SAVE_NONLOCAL); + /* DRAP is needed for stack realign if longjmp is expanded to current + function */ + if (SUPPORTS_STACK_ALIGNMENT) + crtl->need_drap = true; + if (setjmp_alias_set == -1) setjmp_alias_set = new_alias_set (); @@ -1345,7 +1350,7 @@ expand_builtin_apply_args_1 (void) } /* Save the arg pointer to the block. */ - tem = copy_to_reg (virtual_incoming_args_rtx); + tem = copy_to_reg (crtl->args.internal_arg_pointer); #ifdef STACK_GROWS_DOWNWARD /* We need the pointer as the caller actually passed them to us, not as we might have pretended they were passed. Make sure it's a valid @@ -1453,6 +1458,14 @@ expand_builtin_apply (rtx function, rtx arguments, rtx argsize) /* Allocate a block of memory onto the stack and copy the memory arguments to the outgoing arguments address. */ allocate_dynamic_stack_space (argsize, 0, BITS_PER_UNIT); + + /* Set DRAP flag to true, even though allocate_dynamic_stack_space + may have already set current_function_calls_alloca to true. + current_function_calls_alloca won't be set if argsize is zero, + so we have to guarantee need_drap is true here. */ + if (SUPPORTS_STACK_ALIGNMENT) + crtl->need_drap = true; + dest = virtual_outgoing_args_rtx; #ifndef STACK_GROWS_DOWNWARD if (GET_CODE (argsize) == CONST_INT) diff --git a/gcc/caller-save.c b/gcc/caller-save.c index 233caca..e3d76c6 100644 --- a/gcc/caller-save.c +++ b/gcc/caller-save.c @@ -356,10 +356,16 @@ setup_save_areas (void) if (! do_save) continue; - /* We have found an acceptable mode to store in. */ + /* We have found an acceptable mode to store in. Since hard + register is always saved in the widest mode available, + the mode may be wider than necessary, it is OK to reduce + the alignment of spill space. We will verify that it is + equal to or greater than required when we restore and save + the hard register in insert_restore and insert_save. */ regno_save_mem[i][j] - = assign_stack_local (regno_save_mode[i][j], - GET_MODE_SIZE (regno_save_mode[i][j]), 0); + = assign_stack_local_1 (regno_save_mode[i][j], + GET_MODE_SIZE (regno_save_mode[i][j]), + 0, true); /* Setup single word save area just in case... */ for (k = 0; k < j; k++) diff --git a/gcc/calls.c b/gcc/calls.c index 71bdf8c..d48c0d1 100644 --- a/gcc/calls.c +++ b/gcc/calls.c @@ -415,6 +415,10 @@ emit_call_1 (rtx funexp, tree fntree, tree fndecl ATTRIBUTE_UNUSED, rounded_stack_size -= n_popped; rounded_stack_size_rtx = GEN_INT (rounded_stack_size); stack_pointer_delta -= n_popped; + + /* If popup is needed, stack realign must use DRAP */ + if (SUPPORTS_STACK_ALIGNMENT) + crtl->need_drap = true; } if (!ACCUMULATE_OUTGOING_ARGS) @@ -2405,7 +2409,7 @@ expand_call (tree exp, rtx target, int ignore) incoming argument block. */ if (pass == 0) { - argblock = virtual_incoming_args_rtx; + argblock = crtl->args.internal_arg_pointer; argblock #ifdef STACK_GROWS_DOWNWARD = plus_constant (argblock, crtl->args.pretend_args_size); diff --git a/gcc/cfgexpand.c b/gcc/cfgexpand.c index 6040b44..623ad85 100644 --- a/gcc/cfgexpand.c +++ b/gcc/cfgexpand.c @@ -499,10 +499,25 @@ get_decl_align_unit (tree decl) align = DECL_ALIGN (decl); align = LOCAL_ALIGNMENT (TREE_TYPE (decl), align); - if (align > PREFERRED_STACK_BOUNDARY) - align = PREFERRED_STACK_BOUNDARY; + + if (align > MAX_SUPPORTED_STACK_ALIGNMENT) + align = MAX_SUPPORTED_STACK_ALIGNMENT; + + if (SUPPORTS_STACK_ALIGNMENT) + { + if (crtl->stack_alignment_estimated < align) + { + gcc_assert(!crtl->stack_realign_processed); + crtl->stack_alignment_estimated = align; + } + } + + /* stack_alignment_needed > PREFERRED_STACK_BOUNDARY is permitted. + So here we only make sure stack_alignment_needed >= align. */ if (crtl->stack_alignment_needed < align) crtl->stack_alignment_needed = align; + if (crtl->max_used_stack_slot_alignment < crtl->stack_alignment_needed) + crtl->max_used_stack_slot_alignment = crtl->stack_alignment_needed; return align / BITS_PER_UNIT; } @@ -1059,6 +1074,31 @@ defer_stack_allocation (tree var, bool toplevel) static HOST_WIDE_INT expand_one_var (tree var, bool toplevel, bool really_expand) { + if (SUPPORTS_STACK_ALIGNMENT + && TREE_TYPE (var) != error_mark_node + && TREE_CODE (var) == VAR_DECL) + { + unsigned int align; + + /* Because we don't know if VAR will be in register or on stack, + we conservatively assume it will be on stack even if VAR is + eventually put into register after RA pass. For non-automatic + variables, which won't be on stack, we collect alignment of + type and ignore user specified alignment. */ + if (TREE_STATIC (var) || DECL_EXTERNAL (var)) + align = TYPE_ALIGN (TREE_TYPE (var)); + else + align = DECL_ALIGN (var); + + if (crtl->stack_alignment_estimated < align) + { + /* stack_alignment_estimated shouldn't change after stack + realign decision made */ + gcc_assert(!crtl->stack_realign_processed); + crtl->stack_alignment_estimated = align; + } + } + if (TREE_CODE (var) != VAR_DECL) ; else if (DECL_EXTERNAL (var)) @@ -2136,6 +2176,66 @@ discover_nonconstant_array_refs (void) } } +/* This function sets crtl->args.internal_arg_pointer to a virtual + register if DRAP is needed. Local register allocator will replace + virtual_incoming_args_rtx with the virtual register. */ + +static void +expand_stack_alignment (void) +{ + rtx drap_rtx; + unsigned int preferred_stack_boundary; + + if (! SUPPORTS_STACK_ALIGNMENT) + return; + + if (cfun->calls_alloca + || cfun->has_nonlocal_label + || crtl->has_nonlocal_goto) + crtl->need_drap = true; + + gcc_assert (crtl->stack_alignment_needed + <= crtl->stack_alignment_estimated); + + /* Update stack boundary if needed. */ + if (targetm.calls.update_stack_boundary) + targetm.calls.update_stack_boundary (); + + /* Update crtl->stack_alignment_estimated and use it later to align + stack. We check PREFERRED_STACK_BOUNDARY if there may be non-call + exceptions since callgraph doesn't collect incoming stack alignment + in this case. */ + if (flag_non_call_exceptions + && PREFERRED_STACK_BOUNDARY > crtl->preferred_stack_boundary) + preferred_stack_boundary = PREFERRED_STACK_BOUNDARY; + else + preferred_stack_boundary = crtl->preferred_stack_boundary; + if (preferred_stack_boundary > crtl->stack_alignment_estimated) + crtl->stack_alignment_estimated = preferred_stack_boundary; + if (preferred_stack_boundary > crtl->stack_alignment_needed) + crtl->stack_alignment_needed = preferred_stack_boundary; + + crtl->stack_realign_needed + = INCOMING_STACK_BOUNDARY < crtl->stack_alignment_estimated; + + crtl->stack_realign_processed = true; + + /* Target has to redefine TARGET_GET_DRAP_RTX to support stack + alignment. */ + gcc_assert (targetm.calls.get_drap_rtx != NULL); + drap_rtx = targetm.calls.get_drap_rtx (); + + /* Do nothing if NULL is returned, which means DRAP is not needed. */ + if (NULL != drap_rtx) + { + crtl->args.internal_arg_pointer = drap_rtx; + + /* Call fixup_tail_calls to clean up REG_EQUIV note if DRAP is + needed. */ + fixup_tail_calls (); + } +} + /* Translate the intermediate representation contained in the CFG from GIMPLE trees to RTL. @@ -2174,6 +2274,8 @@ gimple_expand_cfg (void) targetm.expand_to_rtl_hook (); crtl->stack_alignment_needed = STACK_BOUNDARY; + crtl->max_used_stack_slot_alignment = STACK_BOUNDARY; + crtl->stack_alignment_estimated = STACK_BOUNDARY; crtl->preferred_stack_boundary = STACK_BOUNDARY; cfun->cfg->max_jumptable_ents = 0; @@ -2248,6 +2350,9 @@ gimple_expand_cfg (void) sbitmap_free (blocks); compact_blocks (); + + expand_stack_alignment (); + #ifdef ENABLE_CHECKING verify_flow_info (); #endif diff --git a/gcc/config/i386/i386-protos.h b/gcc/config/i386/i386-protos.h index 3ebfd3c..3276bd8 100644 --- a/gcc/config/i386/i386-protos.h +++ b/gcc/config/i386/i386-protos.h @@ -28,6 +28,7 @@ extern int ix86_frame_pointer_required (void); extern void ix86_setup_frame_addresses (void); extern void ix86_file_end (void); +extern int ix86_can_eliminate (int, int); extern HOST_WIDE_INT ix86_initial_elimination_offset (int, int); extern void ix86_expand_prologue (void); extern void ix86_expand_epilogue (int); diff --git a/gcc/config/i386/i386.c b/gcc/config/i386/i386.c index dc3854b..c9cbe69 100644 --- a/gcc/config/i386/i386.c +++ b/gcc/config/i386/i386.c @@ -1708,7 +1708,8 @@ static int ix86_regparm; /* -mstackrealign option */ extern int ix86_force_align_arg_pointer; -static const char ix86_force_align_arg_pointer_string[] = "force_align_arg_pointer"; +static const char ix86_force_align_arg_pointer_string[] + = "force_align_arg_pointer"; static rtx (*ix86_gen_leave) (void); static rtx (*ix86_gen_pop1) (rtx); @@ -1717,10 +1718,21 @@ static rtx (*ix86_gen_sub3) (rtx, rtx, rtx); static rtx (*ix86_gen_sub3_carry) (rtx, rtx, rtx, rtx); static rtx (*ix86_gen_one_cmpl2) (rtx, rtx); static rtx (*ix86_gen_monitor) (rtx, rtx, rtx); +static rtx (*ix86_gen_andsp) (rtx, rtx, rtx); /* Preferred alignment for stack boundary in bits. */ unsigned int ix86_preferred_stack_boundary; +/* Alignment for incoming stack boundary in bits specified at + command line. */ +static unsigned int ix86_user_incoming_stack_boundary; + +/* Default alignment for incoming stack boundary in bits. */ +static unsigned int ix86_default_incoming_stack_boundary; + +/* Alignment for incoming stack boundary in bits. */ +unsigned int ix86_incoming_stack_boundary; + /* Values 1-5: see jump.c */ int ix86_branch_cost; @@ -3015,11 +3027,9 @@ override_options (bool main_args_p) if (TARGET_SSE4_2 || TARGET_ABM) ix86_isa_flags |= OPTION_MASK_ISA_POPCNT & ~ix86_isa_flags_explicit; - /* Validate -mpreferred-stack-boundary= value, or provide default. - The default of 128 bits is for Pentium III's SSE __m128. We can't - change it because of optimize_size. Otherwise, we can't mix object - files compiled with -Os and -On. */ - ix86_preferred_stack_boundary = 128; + /* Validate -mpreferred-stack-boundary= value or default it to + PREFERRED_STACK_BOUNDARY_DEFAULT. */ + ix86_preferred_stack_boundary = PREFERRED_STACK_BOUNDARY_DEFAULT; if (ix86_preferred_stack_boundary_string) { i = atoi (ix86_preferred_stack_boundary_string); @@ -3030,6 +3040,31 @@ override_options (bool main_args_p) ix86_preferred_stack_boundary = (1 << i) * BITS_PER_UNIT; } + /* Set the default value for -mstackrealign. */ + if (ix86_force_align_arg_pointer == -1) + ix86_force_align_arg_pointer = STACK_REALIGN_DEFAULT; + + /* Validate -mincoming-stack-boundary= value or default it to + ABI_STACK_BOUNDARY/PREFERRED_STACK_BOUNDARY. */ + if (ix86_force_align_arg_pointer) + ix86_default_incoming_stack_boundary = ABI_STACK_BOUNDARY; + else + ix86_default_incoming_stack_boundary = PREFERRED_STACK_BOUNDARY; + ix86_incoming_stack_boundary = ix86_default_incoming_stack_boundary; + if (ix86_incoming_stack_boundary_string) + { + i = atoi (ix86_incoming_stack_boundary_string); + if (i < (TARGET_64BIT ? 4 : 2) || i > 12) + error ("-mincoming-stack-boundary=%d is not between %d and 12", + i, TARGET_64BIT ? 4 : 2); + else + { + ix86_user_incoming_stack_boundary = (1 << i) * BITS_PER_UNIT; + ix86_incoming_stack_boundary + = ix86_user_incoming_stack_boundary; + } + } + /* Accept -msseregparm only if at least SSE support is enabled. */ if (TARGET_SSEREGPARM && ! TARGET_SSE) @@ -3162,6 +3197,7 @@ override_options (bool main_args_p) ix86_gen_sub3_carry = gen_subdi3_carry_rex64; ix86_gen_one_cmpl2 = gen_one_cmpldi2; ix86_gen_monitor = gen_sse3_monitor64; + ix86_gen_andsp = gen_anddi3; } else { @@ -3172,6 +3208,7 @@ override_options (bool main_args_p) ix86_gen_sub3_carry = gen_subsi3_carry; ix86_gen_one_cmpl2 = gen_one_cmplsi2; ix86_gen_monitor = gen_sse3_monitor; + ix86_gen_andsp = gen_andsi3; } #ifdef USE_IX86_CLD @@ -4002,11 +4039,6 @@ ix86_function_ok_for_sibcall (tree decl, tree exp) && ix86_function_regparm (TREE_TYPE (decl), NULL) >= 3) return false; - /* If we forced aligned the stack, then sibcalling would unalign the - stack, which may break the called function. */ - if (cfun->machine->force_align_arg_pointer) - return false; - /* Otherwise okay. That also includes certain types of indirect calls. */ return true; } @@ -4057,15 +4089,6 @@ ix86_handle_cconv_attribute (tree *node, tree name, *no_add_attrs = true; } - if (!TARGET_64BIT - && lookup_attribute (ix86_force_align_arg_pointer_string, - TYPE_ATTRIBUTES (*node)) - && compare_tree_int (cst, REGPARM_MAX-1)) - { - error ("%s functions limited to %d register parameters", - ix86_force_align_arg_pointer_string, REGPARM_MAX-1); - } - return NULL_TREE; } @@ -4227,8 +4250,7 @@ ix86_function_regparm (const_tree type, const_tree decl) /* We can't use regparm(3) for nested functions as these use static chain pointer in third argument. */ if (local_regparm == 3 - && (decl_function_context (decl) - || ix86_force_align_arg_pointer) + && decl_function_context (decl) && !DECL_NO_STATIC_CHAIN (decl)) local_regparm = 2; @@ -4237,13 +4259,11 @@ ix86_function_regparm (const_tree type, const_tree decl) the callee DECL_STRUCT_FUNCTION is gone, so we fall back to scanning the attributes for the self-realigning property. */ f = DECL_STRUCT_FUNCTION (decl); - if (local_regparm == 3 - && (f ? !!f->machine->force_align_arg_pointer - : !!lookup_attribute (ix86_force_align_arg_pointer_string, - TYPE_ATTRIBUTES (TREE_TYPE (decl))))) - local_regparm = 2; + /* Since current internal arg pointer won't conflict with + parameter passing regs, so no need to change stack + realignment and adjust regparm number. - /* Each fixed register usage increases register pressure, + Each fixed register usage increases register pressure, so less registers should be used for argument passing. This functionality can be overriden by an explicit regparm value. */ @@ -6154,14 +6174,6 @@ setup_incoming_varargs_64 (CUMULATIVE_ARGS *cum) /* Indicate to allocate space on the stack for varargs save area. */ ix86_save_varrargs_registers = 1; - /* We need 16-byte stack alignment to save SSE registers. If user - asked for lower preferred_stack_boundary, lets just hope that he knows - what he is doing and won't varargs SSE values. - - We also may end up assuming that only 64bit values are stored in SSE - register let some floating point program work. */ - if (ix86_preferred_stack_boundary >= BIGGEST_ALIGNMENT) - crtl->stack_alignment_needed = BIGGEST_ALIGNMENT; save_area = frame_pointer_rtx; set = get_varargs_alias_set (); @@ -6344,7 +6356,7 @@ ix86_va_start (tree valist, rtx nextarg) /* Find the overflow area. */ type = TREE_TYPE (ovf); - t = make_tree (type, virtual_incoming_args_rtx); + t = make_tree (type, crtl->args.internal_arg_pointer); if (words != 0) t = build2 (POINTER_PLUS_EXPR, type, t, size_int (words * UNITS_PER_WORD)); @@ -7101,9 +7113,14 @@ ix86_select_alt_pic_regnum (void) if (current_function_is_leaf && !crtl->profile && !ix86_current_function_calls_tls_descriptor) { - int i; + int i, drap; + /* Can't use the same register for both PIC and DRAP. */ + if (crtl->drap_reg) + drap = REGNO (crtl->drap_reg); + else + drap = -1; for (i = 2; i >= 0; --i) - if (!df_regs_ever_live_p (i)) + if (i != drap && !df_regs_ever_live_p (i)) return i; } @@ -7139,8 +7156,8 @@ ix86_save_reg (unsigned int regno, int maybe_eh_return) } } - if (cfun->machine->force_align_arg_pointer - && regno == REGNO (cfun->machine->force_align_arg_pointer)) + if (crtl->drap_reg + && regno == REGNO (crtl->drap_reg)) return 1; return (df_regs_ever_live_p (regno) @@ -7163,6 +7180,24 @@ ix86_nsaved_regs (void) return nregs; } +/* Given FROM and TO register numbers, say whether this elimination is + allowed. If stack alignment is needed, we can only replace argument + pointer with hard frame pointer, or replace frame pointer with stack + pointer. Otherwise, frame pointer elimination is automatically + handled and all other eliminations are valid. */ + +int +ix86_can_eliminate (int from, int to) +{ + if (stack_realign_fp) + return ((from == ARG_POINTER_REGNUM + && to == HARD_FRAME_POINTER_REGNUM) + || (from == FRAME_POINTER_REGNUM + && to == STACK_POINTER_REGNUM)); + else + return to == STACK_POINTER_REGNUM ? !frame_pointer_needed : 1; +} + /* Return the offset between two registers, one to be eliminated, and the other its replacement, at the start of a routine. */ @@ -7206,6 +7241,10 @@ ix86_compute_frame_layout (struct ix86_frame *frame) stack_alignment_needed = crtl->stack_alignment_needed / BITS_PER_UNIT; preferred_alignment = crtl->preferred_stack_boundary / BITS_PER_UNIT; + gcc_assert (!size || stack_alignment_needed); + gcc_assert (preferred_alignment >= STACK_BOUNDARY / BITS_PER_UNIT); + gcc_assert (preferred_alignment <= stack_alignment_needed); + /* During reload iteration the amount of registers saved can change. Recompute the value as needed. Do not recompute when amount of registers didn't change as reload does multiple calls to the function and does not @@ -7248,18 +7287,9 @@ ix86_compute_frame_layout (struct ix86_frame *frame) frame->hard_frame_pointer_offset = offset; - /* Do some sanity checking of stack_alignment_needed and - preferred_alignment, since i386 port is the only using those features - that may break easily. */ - - gcc_assert (!size || stack_alignment_needed); - gcc_assert (preferred_alignment >= STACK_BOUNDARY / BITS_PER_UNIT); - gcc_assert (preferred_alignment <= PREFERRED_STACK_BOUNDARY / BITS_PER_UNIT); - gcc_assert (stack_alignment_needed - <= PREFERRED_STACK_BOUNDARY / BITS_PER_UNIT); - - if (stack_alignment_needed < STACK_BOUNDARY / BITS_PER_UNIT) - stack_alignment_needed = STACK_BOUNDARY / BITS_PER_UNIT; + /* Set offset to aligned because the realigned frame tarts from here. */ + if (stack_realign_fp) + offset = (offset + stack_alignment_needed -1) & -stack_alignment_needed; /* Register save area */ offset += frame->nregs * UNITS_PER_WORD; @@ -7425,38 +7455,131 @@ pro_epilogue_adjust_stack (rtx dest, rtx src, rtx offset, int style) RTX_FRAME_RELATED_P (insn) = 1; } -/* Handle the TARGET_INTERNAL_ARG_POINTER hook. */ +/* Find an available register to be used as dynamic realign argument + pointer regsiter. Such a register will be written in prologue and + used in begin of body, so it must not be + 1. parameter passing register. + 2. GOT pointer. + We reuse static-chain register if it is available. Otherwise, we + use DI for i386 and R13 for x86-64. We chose R13 since it has + shorter encoding. + + Return: the regno of chosen register. */ + +static unsigned int +find_drap_reg (void) +{ + tree decl = cfun->decl; + + if (TARGET_64BIT) + { + /* Use R13 for nested function or function need static chain. + Since function with tail call may use any caller-saved + registers in epilogue, DRAP must not use caller-saved + register in such case. */ + if ((decl_function_context (decl) + && !DECL_NO_STATIC_CHAIN (decl)) + || crtl->tail_call_emit) + return R13_REG; + + return R10_REG; + } + else + { + /* Use DI for nested function or function need static chain. + Since function with tail call may use any caller-saved + registers in epilogue, DRAP must not use caller-saved + register in such case. */ + if ((decl_function_context (decl) + && !DECL_NO_STATIC_CHAIN (decl)) + || crtl->tail_call_emit) + return DI_REG; + + /* Reuse static chain register if it isn't used for parameter + passing. */ + if (ix86_function_regparm (TREE_TYPE (decl), decl) <= 2 + && !lookup_attribute ("fastcall", + TYPE_ATTRIBUTES (TREE_TYPE (decl)))) + return CX_REG; + else + return DI_REG; + } +} + +/* Update incoming stack boundary and estimated stack alignment. */ + +static void +ix86_update_stack_boundary (void) +{ + /* Prefer the one specified at command line. */ + ix86_incoming_stack_boundary + = (ix86_user_incoming_stack_boundary + ? ix86_user_incoming_stack_boundary + : ix86_default_incoming_stack_boundary); + + /* Incoming stack alignment can be changed on individual functions + via force_align_arg_pointer attribute. We use the smallest + incoming stack boundary. */ + if (ix86_incoming_stack_boundary > ABI_STACK_BOUNDARY + && lookup_attribute (ix86_force_align_arg_pointer_string, + TYPE_ATTRIBUTES (TREE_TYPE (current_function_decl)))) + ix86_incoming_stack_boundary = ABI_STACK_BOUNDARY; + + /* Stack at entrance of main is aligned by runtime. We use the + smallest incoming stack boundary. */ + if (ix86_incoming_stack_boundary > MAIN_STACK_BOUNDARY + && DECL_NAME (current_function_decl) + && MAIN_NAME_P (DECL_NAME (current_function_decl)) + && DECL_FILE_SCOPE_P (current_function_decl)) + ix86_incoming_stack_boundary = MAIN_STACK_BOUNDARY; + + /* x86_64 vararg needs 16byte stack alignment for register save + area. */ + if (TARGET_64BIT + && cfun->stdarg + && crtl->stack_alignment_estimated < 128) + crtl->stack_alignment_estimated = 128; +} + +/* Handle the TARGET_GET_DRAP_RTX hook. Return NULL if no DRAP is + needed or an rtx for DRAP otherwise. */ static rtx -ix86_internal_arg_pointer (void) +ix86_get_drap_rtx (void) { - bool has_force_align_arg_pointer = - (0 != lookup_attribute (ix86_force_align_arg_pointer_string, - TYPE_ATTRIBUTES (TREE_TYPE (current_function_decl)))); - if ((FORCE_PREFERRED_STACK_BOUNDARY_IN_MAIN - && DECL_NAME (current_function_decl) - && MAIN_NAME_P (DECL_NAME (current_function_decl)) - && DECL_FILE_SCOPE_P (current_function_decl)) - || ix86_force_align_arg_pointer - || has_force_align_arg_pointer) - { - /* Nested functions can't realign the stack due to a register - conflict. */ - if (DECL_CONTEXT (current_function_decl) - && TREE_CODE (DECL_CONTEXT (current_function_decl)) == FUNCTION_DECL) - { - if (ix86_force_align_arg_pointer) - warning (0, "-mstackrealign ignored for nested functions"); - if (has_force_align_arg_pointer) - error ("%s not supported for nested functions", - ix86_force_align_arg_pointer_string); - return virtual_incoming_args_rtx; - } - cfun->machine->force_align_arg_pointer = gen_rtx_REG (Pmode, CX_REG); - return copy_to_reg (cfun->machine->force_align_arg_pointer); + if (ix86_force_drap || !ACCUMULATE_OUTGOING_ARGS) + crtl->need_drap = true; + + if (stack_realign_drap) + { + /* Assign DRAP to vDRAP and returns vDRAP */ + unsigned int regno = find_drap_reg (); + rtx drap_vreg; + rtx arg_ptr; + rtx seq, insn; + + arg_ptr = gen_rtx_REG (Pmode, regno); + crtl->drap_reg = arg_ptr; + + start_sequence (); + drap_vreg = copy_to_reg (arg_ptr); + seq = get_insns (); + end_sequence (); + + insn = emit_insn_before (seq, NEXT_INSN (entry_of_function ())); + RTX_FRAME_RELATED_P (insn) = 1; + return drap_vreg; } else - return virtual_incoming_args_rtx; + return NULL; +} + +/* Handle the TARGET_INTERNAL_ARG_POINTER hook. */ + +static rtx +ix86_internal_arg_pointer (void) +{ + return virtual_incoming_args_rtx; } /* Handle the TARGET_DWARF_HANDLE_FRAME_UNSPEC hook. @@ -7483,6 +7606,31 @@ ix86_dwarf_handle_frame_unspec (const char *label, rtx pattern, int index) } } +/* Finalize stack_realign_needed flag, which will guide prologue/epilogue + to be generated in correct form. */ +static void +ix86_finalize_stack_realign_flags (void) +{ + /* Check if stack realign is really needed after reload, and + stores result in cfun */ + unsigned int stack_realign = (ix86_incoming_stack_boundary + < (current_function_is_leaf + ? crtl->max_used_stack_slot_alignment + : crtl->stack_alignment_needed)); + + if (crtl->stack_realign_finalized) + { + /* After stack_realign_needed is finalized, we can't no longer + change it. */ + gcc_assert (crtl->stack_realign_needed == stack_realign); + } + else + { + crtl->stack_realign_needed = stack_realign; + crtl->stack_realign_finalized = true; + } +} + /* Expand the prologue into a bunch of separate insns. */ void @@ -7493,52 +7641,58 @@ ix86_expand_prologue (void) struct ix86_frame frame; HOST_WIDE_INT allocate; + ix86_finalize_stack_realign_flags (); + + /* DRAP should not coexist with stack_realign_fp */ + gcc_assert (!(crtl->drap_reg && stack_realign_fp)); + ix86_compute_frame_layout (&frame); - if (cfun->machine->force_align_arg_pointer) + /* Emit prologue code to adjust stack alignment and setup DRAP, in case + of DRAP is needed and stack realignment is really needed after reload */ + if (crtl->drap_reg && crtl->stack_realign_needed) { rtx x, y; + int align_bytes = crtl->stack_alignment_needed / BITS_PER_UNIT; + int param_ptr_offset = (call_used_regs[REGNO (crtl->drap_reg)] + ? 0 : STACK_BOUNDARY / BITS_PER_UNIT); + + gcc_assert (stack_realign_drap); /* Grab the argument pointer. */ - x = plus_constant (stack_pointer_rtx, 4); - y = cfun->machine->force_align_arg_pointer; - insn = emit_insn (gen_rtx_SET (VOIDmode, y, x)); - RTX_FRAME_RELATED_P (insn) = 1; + x = plus_constant (stack_pointer_rtx, + (STACK_BOUNDARY / BITS_PER_UNIT + + param_ptr_offset)); + y = crtl->drap_reg; - /* The unwind info consists of two parts: install the fafp as the cfa, - and record the fafp as the "save register" of the stack pointer. - The later is there in order that the unwinder can see where it - should restore the stack pointer across the and insn. */ - x = gen_rtx_UNSPEC (VOIDmode, gen_rtvec (1, const0_rtx), UNSPEC_DEF_CFA); - x = gen_rtx_SET (VOIDmode, y, x); - RTX_FRAME_RELATED_P (x) = 1; - y = gen_rtx_UNSPEC (VOIDmode, gen_rtvec (1, stack_pointer_rtx), - UNSPEC_REG_SAVE); - y = gen_rtx_SET (VOIDmode, cfun->machine->force_align_arg_pointer, y); - RTX_FRAME_RELATED_P (y) = 1; - x = gen_rtx_PARALLEL (VOIDmode, gen_rtvec (2, x, y)); - x = gen_rtx_EXPR_LIST (REG_FRAME_RELATED_EXPR, x, NULL); - REG_NOTES (insn) = x; + /* Only need to push parameter pointer reg if it is caller + saved reg */ + if (!call_used_regs[REGNO (crtl->drap_reg)]) + { + /* Push arg pointer reg */ + insn = emit_insn (gen_push (y)); + RTX_FRAME_RELATED_P (insn) = 1; + } + + insn = emit_insn (gen_rtx_SET (VOIDmode, y, x)); + RTX_FRAME_RELATED_P (insn) = 1; /* Align the stack. */ - emit_insn (gen_andsi3 (stack_pointer_rtx, stack_pointer_rtx, - GEN_INT (-16))); - - /* And here we cheat like madmen with the unwind info. We force the - cfa register back to sp+4, which is exactly what it was at the - start of the function. Re-pushing the return address results in - the return at the same spot relative to the cfa, and thus is - correct wrt the unwind info. */ - x = cfun->machine->force_align_arg_pointer; - x = gen_frame_mem (Pmode, plus_constant (x, -4)); - insn = emit_insn (gen_push (x)); + insn = emit_insn ((*ix86_gen_andsp) (stack_pointer_rtx, + stack_pointer_rtx, + GEN_INT (-align_bytes))); RTX_FRAME_RELATED_P (insn) = 1; - x = GEN_INT (4); - x = gen_rtx_UNSPEC (VOIDmode, gen_rtvec (1, x), UNSPEC_DEF_CFA); - x = gen_rtx_SET (VOIDmode, stack_pointer_rtx, x); - x = gen_rtx_EXPR_LIST (REG_FRAME_RELATED_EXPR, x, NULL); - REG_NOTES (insn) = x; + /* Replicate the return address on the stack so that return + address can be reached via (argp - 1) slot. This is needed + to implement macro RETURN_ADDR_RTX and intrinsic function + expand_builtin_return_addr etc. */ + x = crtl->drap_reg; + x = gen_frame_mem (Pmode, + plus_constant (x, + -(STACK_BOUNDARY / BITS_PER_UNIT))); + insn = emit_insn (gen_push (x)); + RTX_FRAME_RELATED_P (insn) = 1; } /* Note: AT&T enter does NOT have reversed args. Enter is probably @@ -7553,6 +7707,18 @@ ix86_expand_prologue (void) RTX_FRAME_RELATED_P (insn) = 1; } + if (stack_realign_fp) + { + int align_bytes = crtl->stack_alignment_needed / BITS_PER_UNIT; + gcc_assert (align_bytes > STACK_BOUNDARY / BITS_PER_UNIT); + + /* Align the stack. */ + insn = emit_insn ((*ix86_gen_andsp) (stack_pointer_rtx, + stack_pointer_rtx, + GEN_INT (-align_bytes))); + RTX_FRAME_RELATED_P (insn) = 1; + } + allocate = frame.to_allocate; if (!frame.save_regs_using_mov) @@ -7567,7 +7733,9 @@ ix86_expand_prologue (void) a red zone location */ if (!TARGET_64BIT_MS_ABI && TARGET_RED_ZONE && frame.save_regs_using_mov && (! TARGET_STACK_PROBE || allocate < CHECK_STACK_LIMIT)) - ix86_emit_save_regs_using_mov (frame_pointer_needed ? hard_frame_pointer_rtx + ix86_emit_save_regs_using_mov ((frame_pointer_needed + && !crtl->stack_realign_needed) + ? hard_frame_pointer_rtx : stack_pointer_rtx, -frame.nregs * UNITS_PER_WORD); @@ -7626,8 +7794,11 @@ ix86_expand_prologue (void) && !(!TARGET_64BIT_MS_ABI && TARGET_RED_ZONE && (! TARGET_STACK_PROBE || allocate < CHECK_STACK_LIMIT))) { - if (!frame_pointer_needed || !frame.to_allocate) - ix86_emit_save_regs_using_mov (stack_pointer_rtx, frame.to_allocate); + if (!frame_pointer_needed + || !frame.to_allocate + || crtl->stack_realign_needed) + ix86_emit_save_regs_using_mov (stack_pointer_rtx, + frame.to_allocate); else ix86_emit_save_regs_using_mov (hard_frame_pointer_rtx, -frame.nregs * UNITS_PER_WORD); @@ -7678,6 +7849,16 @@ ix86_expand_prologue (void) emit_insn (gen_blockage ()); } + if (crtl->drap_reg && !crtl->stack_realign_needed) + { + /* vDRAP is setup but after reload it turns out stack realign + isn't necessary, here we will emit prologue to setup DRAP + without stack realign adjustment */ + int drap_bp_offset = STACK_BOUNDARY / BITS_PER_UNIT * 2; + rtx x = plus_constant (hard_frame_pointer_rtx, drap_bp_offset); + insn = emit_insn (gen_rtx_SET (VOIDmode, crtl->drap_reg, x)); + } + /* Emit cld instruction if stringops are used in the function. */ if (TARGET_CLD && ix86_current_function_needs_cld) emit_insn (gen_cld ()); @@ -7719,10 +7900,17 @@ void ix86_expand_epilogue (int style) { int regno; - int sp_valid = !frame_pointer_needed || current_function_sp_is_unchanging; + int sp_valid; struct ix86_frame frame; HOST_WIDE_INT offset; + ix86_finalize_stack_realign_flags (); + + /* When stack is realigned, SP must be valid. */ + sp_valid = (!frame_pointer_needed + || current_function_sp_is_unchanging + || stack_realign_fp); + ix86_compute_frame_layout (&frame); /* Calculate start of saved registers relative to ebp. Special care @@ -7756,11 +7944,16 @@ ix86_expand_epilogue (int style) { /* Restore registers. We can use ebp or esp to address the memory locations. If both are available, default to ebp, since offsets - are known to be small. Only exception is esp pointing directly to the - end of block of saved registers, where we may simplify addressing - mode. */ + are known to be small. Only exception is esp pointing directly + to the end of block of saved registers, where we may simplify + addressing mode. + + If we are realigning stack with bp and sp, regs restore can't + be addressed by bp. sp must be used instead. */ - if (!frame_pointer_needed || (sp_valid && !frame.to_allocate)) + if (!frame_pointer_needed + || (sp_valid && !frame.to_allocate) + || stack_realign_fp) ix86_emit_restore_regs_using_mov (stack_pointer_rtx, frame.to_allocate, style == 2); else @@ -7772,6 +7965,9 @@ ix86_expand_epilogue (int style) { rtx tmp, sa = EH_RETURN_STACKADJ_RTX; + /* Stack align doesn't work with eh_return. */ + gcc_assert (!crtl->stack_realign_needed); + if (frame_pointer_needed) { tmp = gen_rtx_PLUS (Pmode, hard_frame_pointer_rtx, sa); @@ -7813,10 +8009,16 @@ ix86_expand_epilogue (int style) else { /* First step is to deallocate the stack frame so that we can - pop the registers. */ + pop the registers. + + If we realign stack with frame pointer, then stack pointer + won't be able to recover via lea $offset(%bp), %sp, because + there is a padding area between bp and sp for realign. + "add $to_allocate, %sp" must be used instead. */ if (!sp_valid) { gcc_assert (frame_pointer_needed); + gcc_assert (!stack_realign_fp); pro_epilogue_adjust_stack (stack_pointer_rtx, hard_frame_pointer_rtx, GEN_INT (offset), style); @@ -7835,15 +8037,31 @@ ix86_expand_epilogue (int style) if (TARGET_USE_LEAVE) emit_insn ((*ix86_gen_leave) ()); else - emit_insn ((*ix86_gen_pop1) (hard_frame_pointer_rtx)); + { + /* For stack realigned really happens, recover stack + pointer to hard frame pointer is a must, if not using + leave. */ + if (stack_realign_fp) + pro_epilogue_adjust_stack (stack_pointer_rtx, + hard_frame_pointer_rtx, + const0_rtx, style); + emit_insn ((*ix86_gen_pop1) (hard_frame_pointer_rtx)); + } } } - if (cfun->machine->force_align_arg_pointer) + if (crtl->drap_reg && crtl->stack_realign_needed) { - emit_insn (gen_addsi3 (stack_pointer_rtx, - cfun->machine->force_align_arg_pointer, - GEN_INT (-4))); + int param_ptr_offset = (call_used_regs[REGNO (crtl->drap_reg)] + ? 0 : STACK_BOUNDARY / BITS_PER_UNIT); + gcc_assert (stack_realign_drap); + emit_insn ((*ix86_gen_add3) (stack_pointer_rtx, + crtl->drap_reg, + GEN_INT (-(STACK_BOUNDARY / BITS_PER_UNIT + + param_ptr_offset)))); + if (!call_used_regs[REGNO (crtl->drap_reg)]) + emit_insn ((*ix86_gen_pop1) (crtl->drap_reg)); + } /* Sibcall epilogues don't want a return instruction. */ @@ -27275,6 +27493,10 @@ ix86_enum_va_list (int idx, const char **pname, tree *ptree) #define TARGET_PASS_BY_REFERENCE ix86_pass_by_reference #undef TARGET_INTERNAL_ARG_POINTER #define TARGET_INTERNAL_ARG_POINTER ix86_internal_arg_pointer +#undef TARGET_UPDATE_STACK_BOUNDARY +#define TARGET_UPDATE_STACK_BOUNDARY ix86_update_stack_boundary +#undef TARGET_GET_DRAP_RTX +#define TARGET_GET_DRAP_RTX ix86_get_drap_rtx #undef TARGET_DWARF_HANDLE_FRAME_UNSPEC #define TARGET_DWARF_HANDLE_FRAME_UNSPEC ix86_dwarf_handle_frame_unspec #undef TARGET_STRICT_ARGUMENT_NAMING diff --git a/gcc/config/i386/i386.h b/gcc/config/i386/i386.h index c7d33c7..a98e278 100644 --- a/gcc/config/i386/i386.h +++ b/gcc/config/i386/i386.h @@ -648,16 +648,32 @@ enum target_cpu_default #define STACK_BOUNDARY (TARGET_64BIT && DEFAULT_ABI == MS_ABI ? 128 \ : BITS_PER_WORD) +/* Stack boundary of the main function guaranteed by OS. */ +#define MAIN_STACK_BOUNDARY (TARGET_64BIT ? 128 : 32) + +/* Stack boundary guaranteed by ABI. */ +#define ABI_STACK_BOUNDARY (TARGET_64BIT ? 128 : 32) + /* Boundary (in *bits*) on which the stack pointer prefers to be aligned; the compiler cannot rely on having this alignment. */ #define PREFERRED_STACK_BOUNDARY ix86_preferred_stack_boundary -/* As of July 2001, many runtimes do not align the stack properly when - entering main. This causes expand_main_function to forcibly align - the stack, which results in aligned frames for functions called from - main, though it does nothing for the alignment of main itself. */ -#define FORCE_PREFERRED_STACK_BOUNDARY_IN_MAIN \ - (ix86_preferred_stack_boundary > STACK_BOUNDARY && !TARGET_64BIT) +/* It should be ABI_STACK_BOUNDARY. But we set it to 128 bits for + both 32bit and 64bit, to support codes that need 128 bit stack + alignment for SSE instructions, but can't realign the stack. */ +#define PREFERRED_STACK_BOUNDARY_DEFAULT 128 + +/* 1 if -mstackrealign should be turned on by default. It will + generate an alternate prologue and epilogue that realigns the + runtime stack if nessary. This supports mixing codes that keep a + 4-byte aligned stack, as specified by i386 psABI, with codes that + need a 16-byte aligned stack, as required by SSE instructions. If + STACK_REALIGN_DEFAULT is 1 and PREFERRED_STACK_BOUNDARY_DEFAULT is + 128, stacks for all functions may be realigned. */ +#define STACK_REALIGN_DEFAULT 0 + +/* Boundary (in *bits*) on which the incoming stack is aligned. */ +#define INCOMING_STACK_BOUNDARY ix86_incoming_stack_boundary /* Target OS keeps a vector-aligned (128-bit, 16-byte) stack. This is mandatory for the 64-bit ABI, and may or may not be true for other @@ -684,6 +700,9 @@ enum target_cpu_default #define BIGGEST_ALIGNMENT 128 +/* Maximum stack alignment. */ +#define MAX_STACK_ALIGNMENT MAX_OFILE_ALIGNMENT + /* Decide whether a variable of mode MODE should be 128 bit aligned. */ #define ALIGN_MODE_128(MODE) \ ((MODE) == XFmode || SSE_REG_MODE_P (MODE)) @@ -1110,7 +1129,7 @@ do { \ the pic register when possible. The change is visible after the prologue has been emitted. */ -#define REAL_PIC_OFFSET_TABLE_REGNUM 3 +#define REAL_PIC_OFFSET_TABLE_REGNUM BX_REG #define PIC_OFFSET_TABLE_REGNUM \ ((TARGET_64BIT && ix86_cmodel == CM_SMALL_PIC) \ @@ -1629,12 +1648,9 @@ typedef struct ix86_args { { FRAME_POINTER_REGNUM, HARD_FRAME_POINTER_REGNUM}} \ /* Given FROM and TO register numbers, say whether this elimination is - allowed. Frame pointer elimination is automatically handled. - - All other eliminations are valid. */ + allowed. */ -#define CAN_ELIMINATE(FROM, TO) \ - ((TO) == STACK_POINTER_REGNUM ? !frame_pointer_needed : 1) +#define CAN_ELIMINATE(FROM, TO) ix86_can_eliminate ((FROM), (TO)) /* Define the offset between two registers, one to be eliminated, and the other its replacement, at the start of a routine. */ @@ -2203,6 +2219,7 @@ enum asm_dialect { extern enum asm_dialect ix86_asm_dialect; extern unsigned int ix86_preferred_stack_boundary; +extern unsigned int ix86_incoming_stack_boundary; extern int ix86_branch_cost, ix86_section_threshold; /* Smallest class containing REGNO. */ @@ -2304,7 +2321,6 @@ struct machine_function GTY(()) { struct stack_local_entry *stack_locals; const char *some_ld_name; - rtx force_align_arg_pointer; int save_varrargs_registers; int accesses_prev_frame; int optimize_mode_switching[MAX_386_ENTITIES]; diff --git a/gcc/config/i386/i386.md b/gcc/config/i386/i386.md index e7a9827..2e737b1 100644 --- a/gcc/config/i386/i386.md +++ b/gcc/config/i386/i386.md @@ -245,6 +245,7 @@ [(AX_REG 0) (DX_REG 1) (CX_REG 2) + (BX_REG 3) (SI_REG 4) (DI_REG 5) (BP_REG 6) @@ -254,6 +255,7 @@ (FPCR_REG 19) (R10_REG 39) (R11_REG 40) + (R13_REG 42) ]) ;; Insns whose names begin with "x86_" are emitted by gen_FOO calls diff --git a/gcc/config/i386/i386.opt b/gcc/config/i386/i386.opt index d5c0978..fc59b77 100644 --- a/gcc/config/i386/i386.opt +++ b/gcc/config/i386/i386.opt @@ -112,6 +112,10 @@ mfancy-math-387 Target RejectNegative Report InverseMask(NO_FANCY_MATH_387, USE_FANCY_MATH_387) Save Generate sin, cos, sqrt for FPU +mforce-drap +Target Report Var(ix86_force_drap) +Always use Dynamic Realigned Argument Pointer (DRAP) to realign stack + mfp-ret-in-387 Target Report Mask(FLOAT_RETURNS) Save Return values of functions in FPU registers @@ -168,6 +172,10 @@ mpreferred-stack-boundary= Target RejectNegative Joined Var(ix86_preferred_stack_boundary_string) Attempt to keep stack aligned to this power of 2 +mincoming-stack-boundary= +Target RejectNegative Joined Var(ix86_incoming_stack_boundary_string) +Assume incoming stack aligned to this power of 2 + mpush-args Target Report InverseMask(NO_PUSH_ARGS, PUSH_ARGS) Save Use push instructions to save outgoing arguments @@ -193,7 +201,7 @@ Target RejectNegative Mask(SSEREGPARM) Save Use SSE register passing conventions for SF and DF mode mstackrealign -Target Report Var(ix86_force_align_arg_pointer) +Target Report Var(ix86_force_align_arg_pointer) Init(-1) Realign stack in prologue mstack-arg-probe diff --git a/gcc/defaults.h b/gcc/defaults.h index 8029430..e68a065 100644 --- a/gcc/defaults.h +++ b/gcc/defaults.h @@ -563,6 +563,12 @@ along with GCC; see the file COPYING3. If not see #define PREFERRED_STACK_BOUNDARY STACK_BOUNDARY #endif +/* Set INCOMING_STACK_BOUNDARY to PREFERRED_STACK_BOUNDARY if it is not + defined. */ +#ifndef INCOMING_STACK_BOUNDARY +#define INCOMING_STACK_BOUNDARY PREFERRED_STACK_BOUNDARY +#endif + #ifndef TARGET_DEFAULT_PACK_STRUCT #define TARGET_DEFAULT_PACK_STRUCT 0 #endif @@ -950,6 +956,21 @@ along with GCC; see the file COPYING3. If not see #define OUTGOING_REG_PARM_STACK_SPACE(FNTYPE) 0 #endif +/* MAX_STACK_ALIGNMENT is the maximum stack alignment guaranteed by + the backend. MAX_SUPPORTED_STACK_ALIGNMENT is the maximum best + effort stack alignment supported by the backend. If the backend + supports stack alignment, MAX_SUPPORTED_STACK_ALIGNMENT and + MAX_STACK_ALIGNMENT are the same. Otherwise, the incoming stack + boundary will limit the maximum guaranteed stack alignment. */ +#ifdef MAX_STACK_ALIGNMENT +#define MAX_SUPPORTED_STACK_ALIGNMENT MAX_STACK_ALIGNMENT +#else +#define MAX_STACK_ALIGNMENT STACK_BOUNDARY +#define MAX_SUPPORTED_STACK_ALIGNMENT PREFERRED_STACK_BOUNDARY +#endif + +#define SUPPORTS_STACK_ALIGNMENT (MAX_STACK_ALIGNMENT > STACK_BOUNDARY) + #ifndef LOCAL_ALIGNMENT #define LOCAL_ALIGNMENT(TYPE, ALIGNMENT) ALIGNMENT #endif diff --git a/gcc/doc/extend.texi b/gcc/doc/extend.texi index 4c82dc6..9557ba3 100644 --- a/gcc/doc/extend.texi +++ b/gcc/doc/extend.texi @@ -2959,15 +2959,9 @@ floating point arguments on the stack. @cindex @code{force_align_arg_pointer} attribute On the Intel x86, the @code{force_align_arg_pointer} attribute may be applied to individual function definitions, generating an alternate -prologue and epilogue that realigns the runtime stack. This supports -mixing legacy codes that run with a 4-byte aligned stack with modern -codes that keep a 16-byte stack for SSE compatibility. The alternate -prologue and epilogue are slower and bigger than the regular ones, and -the alternate prologue requires a scratch register; this lowers the -number of registers available if used in conjunction with the -@code{regparm} attribute. The @code{force_align_arg_pointer} -attribute is incompatible with nested functions; this is considered a -hard error. +prologue and epilogue that realigns the runtime stack if necessary. +This supports mixing legacy codes that run with a 4-byte aligned stack +with modern codes that keep a 16-byte stack for SSE compatibility. @item resbank @cindex @code{resbank} attribute diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi index bace891..764edc2 100644 --- a/gcc/doc/invoke.texi +++ b/gcc/doc/invoke.texi @@ -555,7 +555,9 @@ Objective-C and Objective-C++ Dialects}. -masm=@var{dialect} -mno-fancy-math-387 @gol -mno-fp-ret-in-387 -msoft-float @gol -mno-wide-multiply -mrtd -malign-double @gol --mpreferred-stack-boundary=@var{num} -mcld -mcx16 -msahf -mrecip @gol +-mpreferred-stack-boundary=@var{num} +-mincoming-stack-boundary=@var{num} +-mcld -mcx16 -msahf -mrecip @gol -mmmx -msse -msse2 -msse3 -mssse3 -msse4.1 -msse4.2 -msse4 @gol -maes -mpclmul @gol -msse4a -m3dnow -mpopcnt -mabm -msse5 @gol @@ -10739,17 +10741,11 @@ when this option is used to set the precision to less than extended precision. @item -mstackrealign @opindex mstackrealign -Realign the stack at entry. On the Intel x86, the -@option{-mstackrealign} option will generate an alternate prologue and -epilogue that realigns the runtime stack. This supports mixing legacy -codes that keep a 4-byte aligned stack with modern codes that keep a -16-byte stack for SSE compatibility. The alternate prologue and -epilogue are slower and bigger than the regular ones, and the -alternate prologue requires an extra scratch register; this lowers the -number of registers available if used in conjunction with the -@code{regparm} attribute. The @option{-mstackrealign} option is -incompatible with the nested function prologue; this is considered a -hard error. See also the attribute @code{force_align_arg_pointer}, +Realign the stack at entry. On the Intel x86, the @option{-mstackrealign} +option will generate an alternate prologue and epilogue that realigns the +runtime stack if necessary. This supports mixing legacy codes that keep +a 4-byte aligned stack with modern codes that keep a 16-byte stack for +SSE compatibility. See also the attribute @code{force_align_arg_pointer}, applicable to individual functions. @item -mpreferred-stack-boundary=@var{num} @@ -10758,6 +10754,12 @@ Attempt to keep the stack boundary aligned to a 2 raised to @var{num} byte boundary. If @option{-mpreferred-stack-boundary} is not specified, the default is 4 (16 bytes or 128 bits). +@item -mincoming-stack-boundary=@var{num} +@opindex mincoming-stack-boundary +Assume the incoming stack is aligned to a 2 raised to @var{num} byte +boundary. If @option{-mincoming-stack-boundary} is not specified, +the one specified by @option{-mpreferred-stack-boundary} will be used. + On Pentium and PentiumPro, @code{double} and @code{long double} values should be aligned to an 8 byte boundary (see @option{-malign-double}) or suffer significant run time performance penalties. On Pentium III, the diff --git a/gcc/doc/tm.texi b/gcc/doc/tm.texi index 5183736..228af27 100644 --- a/gcc/doc/tm.texi +++ b/gcc/doc/tm.texi @@ -1084,6 +1084,12 @@ macro must evaluate to a value equal to or larger than @code{STACK_BOUNDARY}. @end defmac +@defmac INCOMING_STACK_BOUNDARY +Define this macro if the incoming stack boundary may be different +from @code{PREFERRED_STACK_BOUNDARY}. This macro must evaluate +to a value equal to or larger than @code{STACK_BOUNDARY}. +@end defmac + @defmac FUNCTION_BOUNDARY Alignment required for a function entry point, in bits. @end defmac @@ -1122,6 +1128,18 @@ field alignment has not been set by the @code{__attribute__ ((aligned (@var{n})))} construct. @end defmac +@defmac MAX_STACK_ALIGNMENT +Biggest stack alignment guaranteed by the backend. Use this macro +to specify the maximum alignment of a variable on stack. + +If not defined, the default value is @code{STACK_BOUNDARY}. + +@c FIXME: The default should be @code{PREFERRED_STACK_BOUNDARY}. +@c But the fix for PR 32893 indicates that we can only guarantee +@c maximum stack alignment on stack up to @code{STACK_BOUNDARY}, not +@c @code{PREFERRED_STACK_BOUNDARY}, if stack alignment isn't supported. +@end defmac + @defmac MAX_OFILE_ALIGNMENT Biggest alignment supported by the object file format of this machine. Use this macro to limit the alignment which can be specified using the @@ -10586,6 +10604,17 @@ call stack unwinding. It is used in declarations in @file{unwind-generic.h} and the associated definitions of those functions. @end defmac +@deftypefn {Target Hook} void TARGET_UPDATE_STACK_BOUNDARY (void) +Define this macro to update the current function stack boundary if +necessary. +@end deftypefn + +@deftypefn {Target Hook} rtx TARGET_GET_DRAP_RTX (void) +Define this macro to an rtx for Dynamic Realign Argument Pointer if a +different argument pointer register is needed to access the function's +argument list when stack is aligned. +@end deftypefn + @deftypefn {Target Hook} {bool} TARGET_ALLOCATE_STACK_SLOTS_FOR_ARGS (void) When optimization is disabled, this hook indicates whether or not arguments should be allocated to stack slots. Normally, GCC allocates diff --git a/gcc/dwarf2out.c b/gcc/dwarf2out.c index a8e06efb..72514a6 100644 --- a/gcc/dwarf2out.c +++ b/gcc/dwarf2out.c @@ -239,9 +239,18 @@ typedef struct dw_fde_struct GTY(()) bool dw_fde_switched_sections; dw_cfi_ref dw_fde_cfi; unsigned funcdef_number; + HOST_WIDE_INT stack_realignment; + /* Dynamic realign argument pointer register. */ + unsigned int drap_reg; + /* Virtual dynamic realign argument pointer register. */ + unsigned int vdrap_reg; unsigned all_throwers_are_sibcalls : 1; unsigned nothrow : 1; unsigned uses_eh_lsda : 1; + /* Whether we did stack realign in this call frame. */ + unsigned stack_realign : 1; + /* Whether dynamic realign argument pointer register has been saved. */ + unsigned drap_reg_saved: 1; } dw_fde_node; @@ -388,6 +397,8 @@ static void get_cfa_from_loc_descr (dw_cfa_location *, struct dw_loc_descr_struct *); static struct dw_loc_descr_struct *build_cfa_loc (dw_cfa_location *, HOST_WIDE_INT); +static struct dw_loc_descr_struct *build_cfa_aligned_loc + (HOST_WIDE_INT, HOST_WIDE_INT); static void def_cfa_1 (const char *, dw_cfa_location *); /* How to start an assembler comment. */ @@ -621,6 +632,23 @@ static inline void add_cfi (dw_cfi_ref *list_head, dw_cfi_ref cfi) { dw_cfi_ref *p; + dw_fde_ref fde = current_fde (); + + /* When DRAP is used, CFA is defined with an expression. Redefine + CFA may lead to a different CFA value. */ + if (fde && fde->drap_reg != INVALID_REGNUM) + switch (cfi->dw_cfi_opc) + { + case DW_CFA_def_cfa_register: + case DW_CFA_def_cfa_offset: + case DW_CFA_def_cfa_offset_sf: + case DW_CFA_def_cfa: + case DW_CFA_def_cfa_sf: + gcc_unreachable (); + + default: + break; + } /* Find the end of the chain. */ for (p = list_head; (*p) != NULL; p = &(*p)->dw_cfi_next) @@ -880,10 +908,22 @@ static void reg_save (const char *label, unsigned int reg, unsigned int sreg, HOST_WIDE_INT offset) { dw_cfi_ref cfi = new_cfi (); + dw_fde_ref fde = current_fde (); cfi->dw_cfi_oprnd1.dw_cfi_reg_num = reg; - if (sreg == INVALID_REGNUM) + /* When stack is aligned, store REG using DW_CFA_expression with + FP. */ + if (fde + && fde->stack_realign + && sreg == INVALID_REGNUM) + { + cfi->dw_cfi_opc = DW_CFA_expression; + cfi->dw_cfi_oprnd2.dw_cfi_reg_num = reg; + cfi->dw_cfi_oprnd1.dw_cfi_loc + = build_cfa_aligned_loc (offset, fde->stack_realignment); + } + else if (sreg == INVALID_REGNUM) { if (reg & ~0x3f) /* The register number won't fit in 6 bits, so we have to use @@ -1445,6 +1485,11 @@ static dw_cfa_location cfa_temp; difference of the original location and cfa_store's location (or cfa_temp's location if cfa_temp is used). + Rules 16-20: If AND operation happens on sp in prologue, we assume + stack is realigned. We will use a group of DW_OP_XXX + expressions to represent the location of the stored + register instead of CFA+offset. + The Rules "{a,b}" indicates a choice of a xor b. @@ -1538,13 +1583,48 @@ static dw_cfa_location cfa_temp; Rule 15: (set <reg> {unspec, unspec_volatile}) - effects: target-dependent */ + effects: target-dependent + + Rule 16: + (set sp (and: sp <const_int>)) + constraints: cfa_store.reg == sp + effects: current_fde.stack_realign = 1 + cfa_store.offset = 0 + fde->drap_reg = cfa.reg if cfa.reg != sp and cfa.reg != fp + + Rule 17: + (set (mem ({pre_inc, pre_dec} sp)) (mem (plus (cfa.reg) (const_int)))) + effects: cfa_store.offset += -/+ mode_size(mem) + + Rule 18: + (set (mem ({pre_inc, pre_dec} sp)) fp) + constraints: fde->stack_realign == 1 + effects: cfa_store.offset = 0 + cfa.reg != HARD_FRAME_POINTER_REGNUM + + Rule 19: + (set (mem ({pre_inc, pre_dec} sp)) cfa.reg) + constraints: fde->stack_realign == 1 + && cfa.offset == 0 + && cfa.indirect == 0 + && cfa.reg != HARD_FRAME_POINTER_REGNUM + effects: Use DW_CFA_def_cfa_expression to define cfa + cfa.reg == fde->drap_reg + + Rule 20: + (set reg fde->drap_reg) + constraints: fde->vdrap_reg == INVALID_REGNUM + effects: fde->vdrap_reg = reg. + (set mem fde->drap_reg) + constraints: fde->drap_reg_saved == 1 + effects: none. */ static void dwarf2out_frame_debug_expr (rtx expr, const char *label) { rtx src, dest, span; HOST_WIDE_INT offset; + dw_fde_ref fde; /* If RTX_FRAME_RELATED_P is set on a PARALLEL, process each member of the PARALLEL independently. The first element is always processed if @@ -1621,6 +1701,26 @@ dwarf2out_frame_debug_expr (rtx expr, const char *label) src = rsi; } + fde = current_fde (); + + if (GET_CODE (src) == REG + && fde + && fde->drap_reg == REGNO (src) + && (fde->drap_reg_saved + || GET_CODE (dest) == REG)) + { + /* Rule 20 */ + /* If we are saving dynamic realign argument pointer to a + register, the destination is virtual dynamic realign + argument pointer. It may be used to access argument. */ + if (GET_CODE (dest) == REG) + { + gcc_assert (fde->vdrap_reg == INVALID_REGNUM); + fde->vdrap_reg = REGNO (dest); + } + return; + } + switch (GET_CODE (dest)) { case REG: @@ -1649,7 +1749,19 @@ dwarf2out_frame_debug_expr (rtx expr, const char *label) /* For the SPARC and its register window. */ || (DWARF_FRAME_REGNUM (REGNO (src)) == DWARF_FRAME_RETURN_COLUMN)); - queue_reg_save (label, src, dest, 0); + + /* After stack is aligned, we can only save SP in FP + if drap register is used. In this case, we have + to restore stack pointer with the CFA value and we + don't generate this DWARF information. */ + if (fde + && fde->stack_realign + && REGNO (src) == STACK_POINTER_REGNUM) + gcc_assert (REGNO (dest) == HARD_FRAME_POINTER_REGNUM + && fde->drap_reg != INVALID_REGNUM + && cfa.reg != REGNO (src)); + else + queue_reg_save (label, src, dest, 0); } break; @@ -1782,6 +1894,24 @@ dwarf2out_frame_debug_expr (rtx expr, const char *label) targetm.dwarf_handle_frame_unspec (label, expr, XINT (src, 1)); return; + /* Rule 16 */ + case AND: + /* If this AND operation happens on stack pointer in prologue, + we assume the stack is realigned and we extract the + alignment. */ + if (fde && XEXP (src, 0) == stack_pointer_rtx) + { + gcc_assert (cfa_store.reg == REGNO (XEXP (src, 0))); + fde->stack_realign = 1; + fde->stack_realignment = INTVAL (XEXP (src, 1)); + cfa_store.offset = 0; + + if (cfa.reg != STACK_POINTER_REGNUM + && cfa.reg != HARD_FRAME_POINTER_REGNUM) + fde->drap_reg = cfa.reg; + } + return; + default: gcc_unreachable (); } @@ -1790,7 +1920,6 @@ dwarf2out_frame_debug_expr (rtx expr, const char *label) break; case MEM: - gcc_assert (REG_P (src)); /* Saving a register to the stack. Make sure dest is relative to the CFA register. */ @@ -1821,10 +1950,23 @@ dwarf2out_frame_debug_expr (rtx expr, const char *label) if (GET_CODE (XEXP (dest, 0)) == PRE_INC) offset = -offset; - gcc_assert (REGNO (XEXP (XEXP (dest, 0), 0)) == STACK_POINTER_REGNUM + gcc_assert ((REGNO (XEXP (XEXP (dest, 0), 0)) + == STACK_POINTER_REGNUM) && cfa_store.reg == STACK_POINTER_REGNUM); cfa_store.offset += offset; + + /* Rule 18: If stack is aligned, we will use FP as a + reference to represent the address of the stored + regiser. */ + if (fde + && fde->stack_realign + && src == hard_frame_pointer_rtx) + { + gcc_assert (cfa.reg != HARD_FRAME_POINTER_REGNUM); + cfa_store.offset = 0; + } + if (cfa.reg == STACK_POINTER_REGNUM) cfa.offset = cfa_store.offset; @@ -1893,6 +2035,32 @@ dwarf2out_frame_debug_expr (rtx expr, const char *label) if (cfa.offset == 0) { + /* Rule 19 */ + /* If stack is aligned, putting CFA reg into stack means + we can no longer use reg + offset to represent CFA. + Here we use DW_CFA_def_cfa_expression instead. The + result of this expression equals to the original CFA + value. */ + if (fde + && fde->stack_realign + && cfa.indirect == 0 + && cfa.reg != HARD_FRAME_POINTER_REGNUM) + { + dw_cfa_location cfa_exp; + + gcc_assert (fde->drap_reg == cfa.reg); + + cfa_exp.indirect = 1; + cfa_exp.reg = HARD_FRAME_POINTER_REGNUM; + cfa_exp.base_offset = offset; + cfa_exp.offset = 0; + + fde->drap_reg_saved = 1; + + def_cfa_1 (label, &cfa_exp); + break; + } + /* If the source register is exactly the CFA, assume we're saving SP like any other register; this happens on the ARM. */ @@ -1917,6 +2085,12 @@ dwarf2out_frame_debug_expr (rtx expr, const char *label) break; } } + /* Rule 17 */ + /* If the source operand of this MEM operation is not a + register, basically the source is return address. Here + we only care how much stack grew and we don't save it. */ + if (!REG_P (src)) + break; def_cfa_1 (label, &cfa); { @@ -2693,6 +2867,8 @@ dwarf2out_begin_prologue (unsigned int line ATTRIBUTE_UNUSED, fde->nothrow = TREE_NOTHROW (current_function_decl); fde->uses_eh_lsda = crtl->uses_eh_lsda; fde->all_throwers_are_sibcalls = crtl->all_throwers_are_sibcalls; + fde->drap_reg = INVALID_REGNUM; + fde->vdrap_reg = INVALID_REGNUM; args_size = old_args_size = 0; @@ -2924,6 +3100,7 @@ typedef struct dw_loc_list_struct GTY(()) static const char *dwarf_stack_op_name (unsigned); static dw_loc_descr_ref new_loc_descr (enum dwarf_location_atom, unsigned HOST_WIDE_INT, unsigned HOST_WIDE_INT); +static dw_loc_descr_ref int_loc_descriptor (HOST_WIDE_INT); static void add_loc_descr (dw_loc_descr_ref *, dw_loc_descr_ref); static unsigned long size_of_loc_descr (dw_loc_descr_ref); static unsigned long size_of_locs (dw_loc_descr_ref); @@ -3583,6 +3760,9 @@ output_cfa_loc (dw_cfi_ref cfi) dw_loc_descr_ref loc; unsigned long size; + if (cfi->dw_cfi_opc == DW_CFA_expression) + dw2_asm_output_data (1, cfi->dw_cfi_oprnd2.dw_cfi_reg_num, NULL); + /* Output the size of the block. */ loc = cfi->dw_cfi_oprnd1.dw_cfi_loc; size = size_of_locs (loc); @@ -3642,6 +3822,38 @@ build_cfa_loc (dw_cfa_location *cfa, HOST_WIDE_INT offset) return head; } +/* This function builds a dwarf location descriptor sequence for + the address at OFFSET from the CFA when stack is aligned to + ALIGNMENT byte. */ + +static struct dw_loc_descr_struct * +build_cfa_aligned_loc (HOST_WIDE_INT offset, HOST_WIDE_INT alignment) +{ + struct dw_loc_descr_struct *head; + unsigned int dwarf_fp + = DWARF_FRAME_REGNUM (HARD_FRAME_POINTER_REGNUM); + + /* When CFA is defined as FP+OFFSET, emulate stack alignment. */ + if (cfa.reg == HARD_FRAME_POINTER_REGNUM && cfa.indirect == 0) + { + if (dwarf_fp <= 31) + head = new_loc_descr (DW_OP_breg0 + dwarf_fp, 0, 0); + else + head = new_loc_descr (DW_OP_bregx, dwarf_fp, 0); + + add_loc_descr (&head, int_loc_descriptor (alignment)); + add_loc_descr (&head, new_loc_descr (DW_OP_and, 0, 0)); + + add_loc_descr (&head, int_loc_descriptor (offset)); + add_loc_descr (&head, new_loc_descr (DW_OP_plus, 0, 0)); + } + else if (dwarf_fp <= 31) + head = new_loc_descr (DW_OP_breg0 + dwarf_fp, offset, 0); + else + head = new_loc_descr (DW_OP_bregx, dwarf_fp, offset); + return head; +} + /* This function fills in aa dw_cfa_location structure from a dwarf location descriptor sequence. */ @@ -4310,7 +4522,6 @@ static dw_loc_descr_ref one_reg_loc_descriptor (unsigned int, enum var_init_status); static dw_loc_descr_ref multiple_reg_loc_descriptor (rtx, rtx, enum var_init_status); -static dw_loc_descr_ref int_loc_descriptor (HOST_WIDE_INT); static dw_loc_descr_ref based_loc_descr (rtx, HOST_WIDE_INT, enum var_init_status); static int is_based_loc (const_rtx); @@ -8997,6 +9208,10 @@ multiple_reg_loc_descriptor (rtx rtl, rtx regs, return loc_result; } +#endif /* DWARF2_DEBUGGING_INFO */ + +#if defined (DWARF2_DEBUGGING_INFO) || defined (DWARF2_UNWIND_INFO) + /* Return a location descriptor that designates a constant. */ static dw_loc_descr_ref @@ -9035,6 +9250,9 @@ int_loc_descriptor (HOST_WIDE_INT i) return new_loc_descr (op, i, 0); } +#endif + +#ifdef DWARF2_DEBUGGING_INFO /* Return a location descriptor that designates a base+offset location. */ @@ -9044,6 +9262,7 @@ based_loc_descr (rtx reg, HOST_WIDE_INT offset, { unsigned int regno; dw_loc_descr_ref result; + dw_fde_ref fde = current_fde (); /* We only use "frame base" when we're sure we're talking about the post-prologue local stack frame. We do this by *not* running @@ -9060,13 +9279,45 @@ based_loc_descr (rtx reg, HOST_WIDE_INT offset, offset += INTVAL (XEXP (elim, 1)); elim = XEXP (elim, 0); } - gcc_assert (elim == (frame_pointer_needed ? hard_frame_pointer_rtx - : stack_pointer_rtx)); - offset += frame_pointer_fb_offset; + gcc_assert ((SUPPORTS_STACK_ALIGNMENT + && (elim == hard_frame_pointer_rtx + || elim == stack_pointer_rtx)) + || elim == (frame_pointer_needed + ? hard_frame_pointer_rtx + : stack_pointer_rtx)); + + /* If drap register is used to align stack, use frame + pointer + offset to access stack variables. If stack + is aligned without drap, use stack pointer + offset to + access stack variables. */ + if (fde + && fde->stack_realign + && cfa.reg == HARD_FRAME_POINTER_REGNUM + && reg == frame_pointer_rtx) + { + int base_reg + = DWARF_FRAME_REGNUM (cfa.indirect + ? HARD_FRAME_POINTER_REGNUM + : STACK_POINTER_REGNUM); + if (base_reg <= 31) + return new_loc_descr (DW_OP_breg0 + base_reg, offset, 0); + else + return new_loc_descr (DW_OP_bregx, base_reg, offset); + } + offset += frame_pointer_fb_offset; return new_loc_descr (DW_OP_fbreg, offset, 0); } } + else if (fde + && fde->drap_reg != INVALID_REGNUM + && (fde->drap_reg == REGNO (reg) + || fde->vdrap_reg == REGNO (reg))) + { + /* Use cfa+offset to represent the location of arguments passed + on stack when drap is used to align stack. */ + return new_loc_descr (DW_OP_fbreg, offset, 0); + } regno = dbx_reg_number (reg); if (regno <= 31) @@ -11111,8 +11362,13 @@ compute_frame_pointer_to_fb_displacement (HOST_WIDE_INT offset) offset += INTVAL (XEXP (elim, 1)); elim = XEXP (elim, 0); } - gcc_assert (elim == (frame_pointer_needed ? hard_frame_pointer_rtx - : stack_pointer_rtx)); + + gcc_assert ((SUPPORTS_STACK_ALIGNMENT + && (elim == hard_frame_pointer_rtx + || elim == stack_pointer_rtx)) + || elim == (frame_pointer_needed + ? hard_frame_pointer_rtx + : stack_pointer_rtx)); frame_pointer_fb_offset = -offset; } diff --git a/gcc/emit-rtl.c b/gcc/emit-rtl.c index 65fa8e4..a0af7b3 100644 --- a/gcc/emit-rtl.c +++ b/gcc/emit-rtl.c @@ -864,9 +864,18 @@ rtx gen_reg_rtx (enum machine_mode mode) { rtx val; + unsigned int align = GET_MODE_ALIGNMENT (mode); gcc_assert (can_create_pseudo_p ()); + /* If a virtual register with bigger mode alignment is generated, + increase stack alignment estimation because it might be spilled + to stack later. */ + if (SUPPORTS_STACK_ALIGNMENT + && crtl->stack_alignment_estimated < align + && !crtl->stack_realign_processed) + crtl->stack_alignment_estimated = align; + if (generating_concat_p && (GET_MODE_CLASS (mode) == MODE_COMPLEX_FLOAT || GET_MODE_CLASS (mode) == MODE_COMPLEX_INT)) diff --git a/gcc/function.c b/gcc/function.c index ea3ddf4..b9d9ec5 100644 --- a/gcc/function.c +++ b/gcc/function.c @@ -350,10 +350,14 @@ get_stack_local_alignment (tree type, enum machine_mode mode) -2 means use BITS_PER_UNIT, positive specifies alignment boundary in bits. + If REDUCE_ALIGNMENT_OK is true, it is OK to reduce alignment. + We do not round to stack_boundary here. */ rtx -assign_stack_local (enum machine_mode mode, HOST_WIDE_INT size, int align) +assign_stack_local_1 (enum machine_mode mode, HOST_WIDE_INT size, + int align, + bool reduce_alignment_ok ATTRIBUTE_UNUSED) { rtx x, addr; int bigend_correction = 0; @@ -375,17 +379,52 @@ assign_stack_local (enum machine_mode mode, HOST_WIDE_INT size, int align) else alignment = align / BITS_PER_UNIT; + alignment_in_bits = alignment * BITS_PER_UNIT; + if (FRAME_GROWS_DOWNWARD) frame_offset -= size; - /* Ignore alignment we can't do with expected alignment of the boundary. */ - if (alignment * BITS_PER_UNIT > PREFERRED_STACK_BOUNDARY) - alignment = PREFERRED_STACK_BOUNDARY / BITS_PER_UNIT; + /* Ignore alignment if it exceeds MAX_SUPPORTED_STACK_ALIGNMENT. */ + if (alignment_in_bits > MAX_SUPPORTED_STACK_ALIGNMENT) + { + alignment_in_bits = MAX_SUPPORTED_STACK_ALIGNMENT; + alignment = alignment_in_bits / BITS_PER_UNIT; + } - alignment_in_bits = alignment * BITS_PER_UNIT; + if (SUPPORTS_STACK_ALIGNMENT) + { + if (crtl->stack_alignment_estimated < alignment_in_bits) + { + if (!crtl->stack_realign_processed) + crtl->stack_alignment_estimated = alignment_in_bits; + else + { + /* If stack is realigned and stack alignment value + hasn't been finalized, it is OK not to increase + stack_alignment_estimated. The bigger alignment + requirement is recorded in stack_alignment_needed + below. */ + gcc_assert (!crtl->stack_realign_finalized); + if (!crtl->stack_realign_needed) + { + /* It is OK to reduce the alignment as long as the + requested size is 0 or the estimated stack + alignment >= mode alignment. */ + gcc_assert (reduce_alignment_ok + || size == 0 + || (crtl->stack_alignment_estimated + >= GET_MODE_ALIGNMENT (mode))); + alignment_in_bits = crtl->stack_alignment_estimated; + alignment = alignment_in_bits / BITS_PER_UNIT; + } + } + } + } if (crtl->stack_alignment_needed < alignment_in_bits) crtl->stack_alignment_needed = alignment_in_bits; + if (crtl->max_used_stack_slot_alignment < crtl->stack_alignment_needed) + crtl->max_used_stack_slot_alignment = crtl->stack_alignment_needed; /* Calculate how many bytes the start of local variables is off from stack alignment. */ @@ -449,6 +488,14 @@ assign_stack_local (enum machine_mode mode, HOST_WIDE_INT size, int align) return x; } + +/* Wrap up assign_stack_local_1 with last parameter as false. */ + +rtx +assign_stack_local (enum machine_mode mode, HOST_WIDE_INT size, int align) +{ + return assign_stack_local_1 (mode, size, align, false); +} /* Removes temporary slot TEMP from LIST. */ @@ -1167,7 +1214,17 @@ instantiate_new_reg (rtx x, HOST_WIDE_INT *poffset) HOST_WIDE_INT offset; if (x == virtual_incoming_args_rtx) - new = arg_pointer_rtx, offset = in_arg_offset; + { + /* Replace virtual_incoming_args_rtx to internal arg pointer here */ + if (crtl->args.internal_arg_pointer != virtual_incoming_args_rtx) + { + gcc_assert (stack_realign_drap); + new = crtl->args.internal_arg_pointer; + offset = 0; + } + else + new = arg_pointer_rtx, offset = in_arg_offset; + } else if (x == virtual_stack_vars_rtx) new = frame_pointer_rtx, offset = var_offset; else if (x == virtual_stack_dynamic_rtx) @@ -2947,6 +3004,20 @@ assign_parms (tree fndecl) continue; } + /* Estimate stack alignment from parameter alignment. */ + if (SUPPORTS_STACK_ALIGNMENT) + { + unsigned int align = FUNCTION_ARG_BOUNDARY (data.promoted_mode, + data.passed_type); + if (TYPE_ALIGN (data.nominal_type) > align) + align = TYPE_ALIGN (data.passed_type); + if (crtl->stack_alignment_estimated < align) + { + gcc_assert (!crtl->stack_realign_processed); + crtl->stack_alignment_estimated = align; + } + } + if (cfun->stdarg && !TREE_CHAIN (parm)) assign_parms_setup_varargs (&all, &data, false); @@ -2984,6 +3055,28 @@ assign_parms (tree fndecl) now that all parameters have been copied out of hard registers. */ emit_insn (all.first_conversion_insn); + /* Estimate reload stack alignment from scalar return mode. */ + if (SUPPORTS_STACK_ALIGNMENT) + { + if (DECL_RESULT (fndecl)) + { + tree type = TREE_TYPE (DECL_RESULT (fndecl)); + enum machine_mode mode = TYPE_MODE (type); + + if (mode != BLKmode + && mode != VOIDmode + && !AGGREGATE_TYPE_P (type)) + { + unsigned int align = GET_MODE_ALIGNMENT (mode); + if (crtl->stack_alignment_estimated < align) + { + gcc_assert (!crtl->stack_realign_processed); + crtl->stack_alignment_estimated = align; + } + } + } + } + /* If we are receiving a struct value address as the first argument, set up the RTL for the function result. As this might require code to convert the transmitted address to Pmode, we do this here to ensure that possible @@ -3257,15 +3350,43 @@ locate_and_pad_parm (enum machine_mode passed_mode, tree type, int in_regs, = type ? size_in_bytes (type) : size_int (GET_MODE_SIZE (passed_mode)); where_pad = FUNCTION_ARG_PADDING (passed_mode, type); boundary = FUNCTION_ARG_BOUNDARY (passed_mode, type); - if (boundary > PREFERRED_STACK_BOUNDARY) - boundary = PREFERRED_STACK_BOUNDARY; locate->where_pad = where_pad; + + /* Alignment can't exceed MAX_SUPPORTED_STACK_ALIGNMENT. */ + if (boundary > MAX_SUPPORTED_STACK_ALIGNMENT) + boundary = MAX_SUPPORTED_STACK_ALIGNMENT; + locate->boundary = boundary; + if (SUPPORTS_STACK_ALIGNMENT) + { + /* stack_alignment_estimated can't change after stack has been + realigned. */ + if (crtl->stack_alignment_estimated < boundary) + { + if (!crtl->stack_realign_processed) + crtl->stack_alignment_estimated = boundary; + else + { + /* If stack is realigned and stack alignment value + hasn't been finalized, it is OK not to increase + stack_alignment_estimated. The bigger alignment + requirement is recorded in stack_alignment_needed + below. */ + gcc_assert (!crtl->stack_realign_finalized + && crtl->stack_realign_needed); + } + } + } + /* Remember if the outgoing parameter requires extra alignment on the calling function side. */ if (crtl->stack_alignment_needed < boundary) crtl->stack_alignment_needed = boundary; + if (crtl->max_used_stack_slot_alignment < crtl->stack_alignment_needed) + crtl->max_used_stack_slot_alignment = crtl->stack_alignment_needed; + if (crtl->preferred_stack_boundary < boundary) + crtl->preferred_stack_boundary = boundary; #ifdef ARGS_GROW_DOWNWARD locate->slot_offset.constant = -initial_offset_ptr->constant; @@ -4602,7 +4723,8 @@ get_arg_pointer_save_area (void) generated stack slot may not be a valid memory address, so we have to check it and fix it if necessary. */ start_sequence (); - emit_move_insn (validize_mem (ret), virtual_incoming_args_rtx); + emit_move_insn (validize_mem (ret), + crtl->args.internal_arg_pointer); seq = get_insns (); end_sequence (); diff --git a/gcc/function.h b/gcc/function.h index 94f4642..ebba29f 100644 --- a/gcc/function.h +++ b/gcc/function.h @@ -311,6 +311,9 @@ struct rtl_data GTY(()) needed by inner routines. */ rtx x_arg_pointer_save_area; + /* Dynamic Realign Argument Pointer used for realigning stack. */ + rtx drap_reg; + /* Offset to end of allocated area of stack frame. If stack grows down, this is the address of the last stack slot allocated. If stack grows up, this is the address for the next slot. */ @@ -328,12 +331,26 @@ struct rtl_data GTY(()) /* Current nesting level for temporaries. */ int x_temp_slot_level; - /* The largest alignment of slot allocated on the stack. */ + /* The largest alignment needed on the stack, including requirement + for outgoing stack alignment. */ unsigned int stack_alignment_needed; - /* Preferred alignment of the end of stack frame. */ + /* Preferred alignment of the end of stack frame, which is preferred + to call other functions. */ unsigned int preferred_stack_boundary; + /* The largest alignment of slot allocated on the stack. */ + unsigned int max_used_stack_slot_alignment; + + /* The stack alignment estimated before reload, with consideration of + following factors: + 1. Alignment of local stack variables (max_used_stack_slot_alignment) + 2. Alignment requirement to call other functions + (preferred_stack_boundary) + 3. Alignment of non-local stack variables but might be spilled in + local stack. */ + unsigned int stack_alignment_estimated; + /* For reorg. */ /* If some insns can be deferred to the delay slots of the epilogue, the @@ -393,13 +410,32 @@ struct rtl_data GTY(()) /* Nonzero if code to initialize arg_pointer_save_area has been emitted. */ bool arg_pointer_save_area_init; - /* Nonzero means current function must be given a frame pointer. - Set in stmt.c if anything is allocated on the stack there. - Set in reload1.c if anything is allocated on the stack there. */ + /* Nonzero if current function must be given a frame pointer. + Set in global.c if anything is allocated on the stack there. */ bool frame_pointer_needed; /* When set, expand should optimize for speed. */ bool maybe_hot_insn_p; + + /* Nonzero if function stack realignment is needed. This flag may be + set twice: before and after reload. It is set before reload wrt + stack alignment estimation before reload. It will be changed after + reload if by then criteria of stack realignment is different. + The value set after reload is the accurate one and is finalized. */ + bool stack_realign_needed; + + /* Nonzero if function being compiled needs dynamic realigned + argument pointer (drap) if stack needs realigning. */ + bool need_drap; + + /* Nonzero if function stack realignment estimation is done, namely + stack_realign_needed flag has been set before reload wrt + estimated stack alignment info. */ + bool stack_realign_processed; + + /* Nonzero if function stack realignment has been finalized, namely + stack_realign_needed flag has been set and finalized after reload. */ + bool stack_realign_finalized; }; #define return_label (crtl->x_return_label) @@ -414,6 +450,8 @@ struct rtl_data GTY(()) #define temp_slot_level (crtl->x_temp_slot_level) #define nonlocal_goto_handler_labels (crtl->x_nonlocal_goto_handler_labels) #define frame_pointer_needed (crtl->frame_pointer_needed) +#define stack_realign_fp (crtl->stack_realign_needed && !crtl->need_drap) +#define stack_realign_drap (crtl->stack_realign_needed && crtl->need_drap) extern GTY(()) struct rtl_data x_rtl; diff --git a/gcc/global.c b/gcc/global.c index c47033e..690a80c 100644 --- a/gcc/global.c +++ b/gcc/global.c @@ -233,6 +233,7 @@ compute_regsets (HARD_REG_SET *elim_set, = (! flag_omit_frame_pointer || (cfun->calls_alloca && EXIT_IGNORE_STACK) || crtl->accesses_prior_frames + || crtl->stack_realign_needed || FRAME_POINTER_REQUIRED); frame_pointer_needed = need_fp; @@ -256,7 +257,10 @@ compute_regsets (HARD_REG_SET *elim_set, { bool cannot_elim = (! CAN_ELIMINATE (eliminables[i].from, eliminables[i].to) - || (eliminables[i].to == STACK_POINTER_REGNUM && need_fp)); + || (eliminables[i].to == STACK_POINTER_REGNUM + && need_fp + && (! SUPPORTS_STACK_ALIGNMENT + || ! stack_realign_fp))); if (!regs_asm_clobbered[eliminables[i].from]) { diff --git a/gcc/reload1.c b/gcc/reload1.c index f28b01c..9b81062 100644 --- a/gcc/reload1.c +++ b/gcc/reload1.c @@ -3663,8 +3663,11 @@ update_eliminables (HARD_REG_SET *pset) frame_pointer_needed = 1; for (ep = reg_eliminate; ep < ®_eliminate[NUM_ELIMINABLE_REGS]; ep++) { - if (ep->can_eliminate && ep->from == FRAME_POINTER_REGNUM - && ep->to != HARD_FRAME_POINTER_REGNUM) + if (ep->can_eliminate + && ep->from == FRAME_POINTER_REGNUM + && ep->to != HARD_FRAME_POINTER_REGNUM + && (! SUPPORTS_STACK_ALIGNMENT + || ! crtl->stack_realign_needed)) frame_pointer_needed = 0; if (! ep->can_eliminate && ep->can_eliminate_previous) @@ -3720,7 +3723,10 @@ init_elim_table (void) ep->to = ep1->to; ep->can_eliminate = ep->can_eliminate_previous = (CAN_ELIMINATE (ep->from, ep->to) - && ! (ep->to == STACK_POINTER_REGNUM && frame_pointer_needed)); + && ! (ep->to == STACK_POINTER_REGNUM + && frame_pointer_needed + && (! SUPPORTS_STACK_ALIGNMENT + || ! stack_realign_fp))); } #else reg_eliminate[0].from = reg_eliminate_1[0].from; @@ -1572,6 +1572,7 @@ extern rtx simplify_subtraction (rtx); /* In function.c */ extern rtx assign_stack_local (enum machine_mode, HOST_WIDE_INT, int); +extern rtx assign_stack_local_1 (enum machine_mode, HOST_WIDE_INT, int, bool); extern rtx assign_stack_temp (enum machine_mode, HOST_WIDE_INT, int); extern rtx assign_stack_temp_for_type (enum machine_mode, HOST_WIDE_INT, int, tree); @@ -1813,7 +1813,7 @@ expand_nl_goto_receiver (void) { /* Now restore our arg pointer from the address at which it was saved in our stack frame. */ - emit_move_insn (virtual_incoming_args_rtx, + emit_move_insn (crtl->args.internal_arg_pointer, copy_to_reg (get_arg_pointer_save_area ())); } } diff --git a/gcc/target-def.h b/gcc/target-def.h index bf8c7e7..cd64b38 100644 --- a/gcc/target-def.h +++ b/gcc/target-def.h @@ -571,6 +571,8 @@ #define TARGET_FUNCTION_VALUE default_function_value #define TARGET_INTERNAL_ARG_POINTER default_internal_arg_pointer +#define TARGET_UPDATE_STACK_BOUNDARY NULL +#define TARGET_GET_DRAP_RTX NULL #define TARGET_ALLOCATE_STACK_SLOTS_FOR_ARGS hook_bool_void_true #define TARGET_CALLS { \ @@ -592,6 +594,8 @@ TARGET_INVALID_ARG_FOR_UNPROTOTYPED_FN, \ TARGET_FUNCTION_VALUE, \ TARGET_INTERNAL_ARG_POINTER, \ + TARGET_UPDATE_STACK_BOUNDARY, \ + TARGET_GET_DRAP_RTX, \ TARGET_ALLOCATE_STACK_SLOTS_FOR_ARGS \ } diff --git a/gcc/target.h b/gcc/target.h index 331cc53..3a104c5 100644 --- a/gcc/target.h +++ b/gcc/target.h @@ -839,6 +839,13 @@ struct gcc_target current function. */ rtx (*internal_arg_pointer) (void); + /* Update the current function stack boundary if needed. */ + void (*update_stack_boundary) (void); + + /* Handle stack alignment and return an rtx for Dynamic Realign + Argument Pointer if necessary. */ + rtx (*get_drap_rtx) (void); + /* Return true if all function parameters should be spilled to the stack. */ bool (*allocate_stack_slots_for_args) (void); diff --git a/gcc/tree-vectorizer.c b/gcc/tree-vectorizer.c index 3526de4..22e2e0c 100644 --- a/gcc/tree-vectorizer.c +++ b/gcc/tree-vectorizer.c @@ -1830,9 +1830,7 @@ vect_can_force_dr_alignment_p (const_tree decl, unsigned int alignment) if (TREE_STATIC (decl)) return (alignment <= MAX_OFILE_ALIGNMENT); else - /* This used to be PREFERRED_STACK_BOUNDARY, however, that is not 100% - correct until someone implements forced stack alignment. */ - return (alignment <= STACK_BOUNDARY); + return (alignment <= MAX_STACK_ALIGNMENT); } |