i386.md (@leave_<mode>): New expander.

	* config/i386/i386.md (@leave_<mode>): New expander.
	(*leave): Rename from leave.
	(*leave_rex64): Rename from leave_rex64.
	(@monitorx_<mode>): Rename from monitorx_<mode>.
	(@clzero_<mode>): Rename from clzero_<mode>.
	* config/i386/sse.md (@sse3_monitor_<mode>): Rename from
	sse3_monitor_<mode>.
	* config/i386/i386.c (*ix86_gen_leave): Remove indirect function.
	(*ix86_gen_monitor): Ditto.
	(*ix86_gen_monitorx): Ditto.
	(*ix86_gen_clzero): Ditto.
	(*ix86_gen_one_cmpl2): Ditto.
	(ix86_emit_leave): Use gen_leave instead of ix86_gen_leave.
	* config/i386/i386-expand.c (ix86_expand_builtin)
	<case IX86_BUILTIN_MONITOR>: Use gen_sse3_monitor
	instead of ix86_gen_monitor.
	<case IX86_BUILTIN_MONITORX>: Use gen_monitorx
	instead of ix86_gen_monitorx.
	<case IX86_BUILTIN_CLZERO>: Use gen_clzero
	instead of ix86_gen_clzero.
	* config/i386/i386-options.c (ix86_option_override_internal):
	Do not initialize ix86_gen_leave, ix86_gen_monitor,
	ix86_gen_monitorx, ix86_gen_clzero and ix86_gen_one_cmpl2.

	* config/i386/i386.md (@tls_global_dynamic_64_<mode>):
	Rename from tls_global_dynamic_64_<mode>.
	(@tls_local_dynamic_base_64_<mode>): Rename from
	tls_local_dynamic_base_64_<mode>.
	* config/i386/i386.c (*ix86_gen_tls_global_dynamic_64):
	Remove indirect function.
	(*ix86_gen_tls_local_dynamic_base_64): Ditto.
	(legitimize_tls_address): Use gen_tls_global_dynamic_64 function
	instead of ix86_gen_tls_global_dynamic_64.
	Use gen_tls_local_dynamic_base_64 instead of
	ix86_gen_tls_local_dynamic_base_64.
	* config/i386/i386-options.c (ix86_option_override_internal):
	Do not initialize ix86_gen_tls_global_dynamic_64 and
	ix86_gen_tls_local_dynamic_base_64.

	* config/i386/i386.md (@pro_epilogue_adjust_stack_add_<mode>)
	Rename from pro_epilogue_adjust_stack_<mode>_add.
	(@pro_epilogue_adjust_stack_sub_<mode>)
	Rename from pro_epilogue_adjust_stack_<mode>_sub.
	(@allocate_stack_worker_probe_<mode>):
	Rename from allocate_stack_worker_probe_<mode>.
	(allocate_stack): Use gen_allocate_stack_worker_probe.
	(probe_stack): Use gen_probe_stack_1.
	(@probe_stack_1_<mode>): Rename from probe_stack_<mode>.
	(@adjust_stack_and_probe_<mode>): Rename from
	adjust_stack_and_probe<mode>.
	(@probe_stack_range_<mode>): Rename from probe_stack_range<mode>.
	(stack_protect_set): Use gen_stack_protect_set_1.
	(@stack_protect_set_1_<mode>): Rename from stack_protect_set_<mode>.
	(stack_protect_test): Use gen_stack_protect_test_1.
	(@stack_protect_test_1_<mode>): Rename from stack_protect_test_<mode>.
	* config/i386/i386.c (*ix86_gen_allocate_stack_worker):
	Remove indirect function.
	(*ix86_gen_adjust_stack_and_probe): Ditto.
	(*ix86_gen_probe_stack_range): Ditto.
	(pro_epilogue_adjust_stack): Use gen_pro_epilogue_adjust_stack_add
	instead of gen_pro_epilogue_adjust_stack_{si,di}_add.
	(ix86_adjust_stack_and_probe_stack_clash): Use
	gen_adjust_stack_and_probe instead of ix86_gen_adjust_stack_and_probe.
	(ix86_adjust_stack_and_probe): Ditto.
	(ix86_emit_probe_stack_range): Use gen_probe_stack_range instead
	of ix86_gen_probe_stack_range.
	(ix86_expand_prologue):  Use gen_pro_epilogue_adjust_stack_sub
	instead of gen_pro_epilogue_adjust_stack_{si,di}_sub.
	* config/i386/x86-tune-sched.c (ix86_macro_fusion_pair_p):
	Include insn-opinit.h.  Use code_for_stack_protect_test_1 instead of
	CODE_FOR_stack_protect_test_{si,di}.
	* config/i386/i386-options.c (ix86_option_override_internal):
	Do not initialize ix86_gen_allocate_stack_worker,
	ix86_gen_adjust_stack_and_probe and ix86_gen_probe_stack_range.

From-SVN: r271636

8 files changed, 140 insertions, 129 deletions

diff --git a/gcc/ChangeLog b/gcc/ChangeLog
index 2ffb48c..67e8333 100644
--- a/gcc/ChangeLog
+++ b/gcc/ChangeLog
@@ -1,3 +1,84 @@
+2019-05-26  Uroš Bizjak  <ubizjak@gmail.com>
+
+	* config/i386/i386.md (@leave_<mode>): New expander.
+	(*leave): Rename from leave.
+	(*leave_rex64): Rename from leave_rex64.
+	(@monitorx_<mode>): Rename from monitorx_<mode>.
+	(@clzero_<mode>): Rename from clzero_<mode>.
+	* config/i386/sse.md (@sse3_monitor_<mode>): Rename from
+	sse3_monitor_<mode>.
+	* config/i386/i386.c (*ix86_gen_leave): Remove indirect function.
+	(*ix86_gen_monitor): Ditto.
+	(*ix86_gen_monitorx): Ditto.
+	(*ix86_gen_clzero): Ditto.
+	(*ix86_gen_one_cmpl2): Ditto.
+	(ix86_emit_leave): Use gen_leave instead of ix86_gen_leave.
+	* config/i386/i386-expand.c (ix86_expand_builtin)
+	<case IX86_BUILTIN_MONITOR>: Use gen_sse3_monitor
+	instead of ix86_gen_monitor.
+	<case IX86_BUILTIN_MONITORX>: Use gen_monitorx
+	instead of ix86_gen_monitorx.
+	<case IX86_BUILTIN_CLZERO>: Use gen_clzero
+	instead of ix86_gen_clzero.
+	* config/i386/i386-options.c (ix86_option_override_internal):
+	Do not initialize ix86_gen_leave, ix86_gen_monitor,
+	ix86_gen_monitorx, ix86_gen_clzero and ix86_gen_one_cmpl2.
+
+2019-05-26  Uroš Bizjak  <ubizjak@gmail.com>
+
+	* config/i386/i386.md (@tls_global_dynamic_64_<mode>):
+	Rename from tls_global_dynamic_64_<mode>.
+	(@tls_local_dynamic_base_64_<mode>): Rename from
+	tls_local_dynamic_base_64_<mode>.
+	* config/i386/i386.c (*ix86_gen_tls_global_dynamic_64):
+	Remove indirect function.
+	(*ix86_gen_tls_local_dynamic_base_64): Ditto.
+	(legitimize_tls_address): Use gen_tls_global_dynamic_64 function
+	instead of ix86_gen_tls_global_dynamic_64.
+	Use gen_tls_local_dynamic_base_64 instead of
+	ix86_gen_tls_local_dynamic_base_64.
+	* config/i386/i386-options.c (ix86_option_override_internal):
+	Do not initialize ix86_gen_tls_global_dynamic_64 and
+	ix86_gen_tls_local_dynamic_base_64.
+
+2019-05-26  Uroš Bizjak  <ubizjak@gmail.com>
+
+	* config/i386/i386.md (@pro_epilogue_adjust_stack_add_<mode>)
+	Rename from pro_epilogue_adjust_stack_<mode>_add.
+	(@pro_epilogue_adjust_stack_sub_<mode>)
+	Rename from pro_epilogue_adjust_stack_<mode>_sub.
+	(@allocate_stack_worker_probe_<mode>):
+	Rename from allocate_stack_worker_probe_<mode>.
+	(allocate_stack): Use gen_allocate_stack_worker_probe.
+	(probe_stack): Use gen_probe_stack_1.
+	(@probe_stack_1_<mode>): Rename from probe_stack_<mode>.
+	(@adjust_stack_and_probe_<mode>): Rename from
+	adjust_stack_and_probe<mode>.
+	(@probe_stack_range_<mode>): Rename from probe_stack_range<mode>.
+	(stack_protect_set): Use gen_stack_protect_set_1.
+	(@stack_protect_set_1_<mode>): Rename from stack_protect_set_<mode>.
+	(stack_protect_test): Use gen_stack_protect_test_1.
+	(@stack_protect_test_1_<mode>): Rename from stack_protect_test_<mode>.
+	* config/i386/i386.c (*ix86_gen_allocate_stack_worker):
+	Remove indirect function.
+	(*ix86_gen_adjust_stack_and_probe): Ditto.
+	(*ix86_gen_probe_stack_range): Ditto.
+	(pro_epilogue_adjust_stack): Use gen_pro_epilogue_adjust_stack_add
+	instead of gen_pro_epilogue_adjust_stack_{si,di}_add.
+	(ix86_adjust_stack_and_probe_stack_clash): Use
+	gen_adjust_stack_and_probe instead of ix86_gen_adjust_stack_and_probe.
+	(ix86_adjust_stack_and_probe): Ditto.
+	(ix86_emit_probe_stack_range): Use gen_probe_stack_range instead
+	of ix86_gen_probe_stack_range.
+	(ix86_expand_prologue):  Use gen_pro_epilogue_adjust_stack_sub
+	instead of gen_pro_epilogue_adjust_stack_{si,di}_sub.
+	* config/i386/x86-tune-sched.c (ix86_macro_fusion_pair_p):
+	Include insn-opinit.h.  Use code_for_stack_protect_test_1 instead of
+	CODE_FOR_stack_protect_test_{si,di}.
+	* config/i386/i386-options.c (ix86_option_override_internal):
+	Do not initialize ix86_gen_allocate_stack_worker,
+	ix86_gen_adjust_stack_and_probe and ix86_gen_probe_stack_range.
+
 2019-05-26  Gerald Pfeifer  <gerald@pfeifer.com>
 
 	* doc/invoke.texi (Link Options): Many editorial changes around
diff --git a/gcc/config/i386/i386-expand.c b/gcc/config/i386/i386-expand.c
index 87e0973..06ed107 100644
--- a/gcc/config/i386/i386-expand.c
+++ b/gcc/config/i386/i386-expand.c
@@ -11067,8 +11067,8 @@ ix86_expand_builtin (tree exp, rtx target, rtx subtarget,
 	op2 = copy_to_mode_reg (SImode, op2);
 
       emit_insn (fcode == IX86_BUILTIN_MONITOR 
-		 ? ix86_gen_monitor (op0, op1, op2)
-		 : ix86_gen_monitorx (op0, op1, op2));
+		 ? gen_sse3_monitor (Pmode, op0, op1, op2)
+		 : gen_monitorx (Pmode, op0, op1, op2));
       return 0;
 
     case IX86_BUILTIN_MWAIT:
@@ -11180,7 +11180,7 @@ ix86_expand_builtin (tree exp, rtx target, rtx subtarget,
       op0 = expand_normal (arg0);
       if (!REG_P (op0))
 	op0 = ix86_zero_extend_to_Pmode (op0);
-      emit_insn (ix86_gen_clzero (op0));
+      emit_insn (gen_clzero (Pmode, op0));
       return 0;
 
     case IX86_BUILTIN_CLDEMOTE:
diff --git a/gcc/config/i386/i386-options.c b/gcc/config/i386/i386-options.c
index 0f23662..5f14bb5 100644
--- a/gcc/config/i386/i386-options.c
+++ b/gcc/config/i386/i386-options.c
@@ -2552,52 +2552,19 @@ ix86_option_override_internal (bool main_args_p,
   if (!TARGET_64BIT_P (opts->x_ix86_isa_flags) && !opts->x_flag_split_stack)
     targetm.expand_builtin_va_start = NULL;
 
-  if (TARGET_64BIT_P (opts->x_ix86_isa_flags))
-    {
-      ix86_gen_leave = gen_leave_rex64;
-      if (Pmode == DImode)
-	{
-	  ix86_gen_tls_global_dynamic_64 = gen_tls_global_dynamic_64_di;
-	  ix86_gen_tls_local_dynamic_base_64
-	    = gen_tls_local_dynamic_base_64_di;
-	}
-      else
-	{
-	  ix86_gen_tls_global_dynamic_64 = gen_tls_global_dynamic_64_si;
-	  ix86_gen_tls_local_dynamic_base_64
-	    = gen_tls_local_dynamic_base_64_si;
-	}
-    }
-  else
-    ix86_gen_leave = gen_leave;
-
   if (Pmode == DImode)
     {
       ix86_gen_add3 = gen_adddi3;
       ix86_gen_sub3 = gen_subdi3;
       ix86_gen_sub3_carry = gen_subdi3_carry;
-      ix86_gen_one_cmpl2 = gen_one_cmpldi2;
       ix86_gen_andsp = gen_anddi3;
-      ix86_gen_allocate_stack_worker = gen_allocate_stack_worker_probe_di;
-      ix86_gen_adjust_stack_and_probe = gen_adjust_stack_and_probedi;
-      ix86_gen_probe_stack_range = gen_probe_stack_rangedi;
-      ix86_gen_monitor = gen_sse3_monitor_di;
-      ix86_gen_monitorx = gen_monitorx_di;
-      ix86_gen_clzero = gen_clzero_di;
     }
   else
     {
       ix86_gen_add3 = gen_addsi3;
       ix86_gen_sub3 = gen_subsi3;
       ix86_gen_sub3_carry = gen_subsi3_carry;
-      ix86_gen_one_cmpl2 = gen_one_cmplsi2;
       ix86_gen_andsp = gen_andsi3;
-      ix86_gen_allocate_stack_worker = gen_allocate_stack_worker_probe_si;
-      ix86_gen_adjust_stack_and_probe = gen_adjust_stack_and_probesi;
-      ix86_gen_probe_stack_range = gen_probe_stack_rangesi;
-      ix86_gen_monitor = gen_sse3_monitor_si;
-      ix86_gen_monitorx = gen_monitorx_si;
-      ix86_gen_clzero = gen_clzero_si;
     }
 
 #ifdef USE_IX86_CLD
diff --git a/gcc/config/i386/i386-options.h b/gcc/config/i386/i386-options.h
index 817ddda..5021ac5 100644
--- a/gcc/config/i386/i386-options.h
+++ b/gcc/config/i386/i386-options.h
@@ -55,20 +55,10 @@ extern tree (*ix86_veclib_handler) (combined_fn, tree, tree);
 extern tree ix86_veclibabi_svml (combined_fn, tree, tree);
 extern tree ix86_veclibabi_acml (combined_fn, tree, tree);
 
-extern rtx (*ix86_gen_leave) (void);
 extern rtx (*ix86_gen_add3) (rtx, rtx, rtx);
 extern rtx (*ix86_gen_sub3) (rtx, rtx, rtx);
 extern rtx (*ix86_gen_sub3_carry) (rtx, rtx, rtx, rtx, rtx);
-extern rtx (*ix86_gen_one_cmpl2) (rtx, rtx);
-extern rtx (*ix86_gen_monitor) (rtx, rtx, rtx);
-extern rtx (*ix86_gen_monitorx) (rtx, rtx, rtx);
-extern rtx (*ix86_gen_clzero) (rtx);
 extern rtx (*ix86_gen_andsp) (rtx, rtx, rtx);
-extern rtx (*ix86_gen_allocate_stack_worker) (rtx, rtx);
-extern rtx (*ix86_gen_adjust_stack_and_probe) (rtx, rtx, rtx);
-extern rtx (*ix86_gen_probe_stack_range) (rtx, rtx, rtx);
-extern rtx (*ix86_gen_tls_global_dynamic_64) (rtx, rtx, rtx);
-extern rtx (*ix86_gen_tls_local_dynamic_base_64) (rtx, rtx);
 
 enum ix86_function_specific_strings
 {
diff --git a/gcc/config/i386/i386.c b/gcc/config/i386/i386.c
index 79fcb5c..75ad105 100644
--- a/gcc/config/i386/i386.c
+++ b/gcc/config/i386/i386.c
@@ -349,20 +349,10 @@ enum processor_type ix86_arch;
 /* True if processor has SSE prefetch instruction.  */
 unsigned char x86_prefetch_sse;
 
-rtx (*ix86_gen_leave) (void);
 rtx (*ix86_gen_add3) (rtx, rtx, rtx);
 rtx (*ix86_gen_sub3) (rtx, rtx, rtx);
 rtx (*ix86_gen_sub3_carry) (rtx, rtx, rtx, rtx, rtx);
-rtx (*ix86_gen_one_cmpl2) (rtx, rtx);
-rtx (*ix86_gen_monitor) (rtx, rtx, rtx);
-rtx (*ix86_gen_monitorx) (rtx, rtx, rtx);
-rtx (*ix86_gen_clzero) (rtx);
 rtx (*ix86_gen_andsp) (rtx, rtx, rtx);
-rtx (*ix86_gen_allocate_stack_worker) (rtx, rtx);
-rtx (*ix86_gen_adjust_stack_and_probe) (rtx, rtx, rtx);
-rtx (*ix86_gen_probe_stack_range) (rtx, rtx, rtx);
-rtx (*ix86_gen_tls_global_dynamic_64) (rtx, rtx, rtx);
-rtx (*ix86_gen_tls_local_dynamic_base_64) (rtx, rtx);
 
 /* Preferred alignment for stack boundary in bits.  */
 unsigned int ix86_preferred_stack_boundary;
@@ -6551,34 +6541,29 @@ pro_epilogue_adjust_stack (rtx dest, rtx src, rtx offset,
 			   int style, bool set_cfa)
 {
   struct machine_function *m = cfun->machine;
+  rtx addend = offset;
   rtx insn;
   bool add_frame_related_expr = false;
 
-  if (Pmode == SImode)
-    insn = gen_pro_epilogue_adjust_stack_si_add (dest, src, offset);
-  else if (x86_64_immediate_operand (offset, DImode))
-    insn = gen_pro_epilogue_adjust_stack_di_add (dest, src, offset);
-  else
+  if (!x86_64_immediate_operand (offset, Pmode))
     {
-      rtx tmp;
       /* r11 is used by indirect sibcall return as well, set before the
 	 epilogue and used after the epilogue.  */
       if (style)
-        tmp = gen_rtx_REG (DImode, R11_REG);
+        addend = gen_rtx_REG (Pmode, R11_REG);
       else
 	{
 	  gcc_assert (src != hard_frame_pointer_rtx
 		      && dest != hard_frame_pointer_rtx);
-	  tmp = hard_frame_pointer_rtx;
+	  addend = hard_frame_pointer_rtx;
 	}
-      insn = emit_insn (gen_rtx_SET (tmp, offset));
+      emit_insn (gen_rtx_SET (addend, offset));
       if (style < 0)
 	add_frame_related_expr = true;
-
-      insn = gen_pro_epilogue_adjust_stack_di_add (dest, src, tmp);
     }
 
-  insn = emit_insn (insn);
+  insn = emit_insn (gen_pro_epilogue_adjust_stack_add
+		    (Pmode, dest, src, addend));
   if (style >= 0)
     ix86_add_queued_cfa_restore_notes (insn);
 
@@ -7078,8 +7063,8 @@ ix86_adjust_stack_and_probe_stack_clash (HOST_WIDE_INT size,
 
       /* Step 3: the loop.  */
       rtx size_rtx = GEN_INT (rounded_size);
-      insn = emit_insn (ix86_gen_adjust_stack_and_probe (sr.reg, sr.reg,
-							 size_rtx));
+      insn = emit_insn (gen_adjust_stack_and_probe (Pmode, sr.reg, sr.reg,
+						    size_rtx));
       if (m->fs.cfa_reg == stack_pointer_rtx)
 	{
 	  m->fs.cfa_offset += rounded_size;
@@ -7232,7 +7217,7 @@ ix86_adjust_stack_and_probe (HOST_WIDE_INT size,
 	 adjusts SP and probes to PROBE_INTERVAL + N * PROBE_INTERVAL for
 	 values of N from 1 until it is equal to ROUNDED_SIZE.  */
 
-      emit_insn (ix86_gen_adjust_stack_and_probe (sr.reg, sr.reg, size_rtx));
+      emit_insn (gen_adjust_stack_and_probe (Pmode, sr.reg, sr.reg, size_rtx));
 
 
       /* Step 4: adjust SP and probe at PROBE_INTERVAL + SIZE if we cannot
@@ -7390,7 +7375,8 @@ ix86_emit_probe_stack_range (HOST_WIDE_INT first, HOST_WIDE_INT size,
          probes at FIRST + N * PROBE_INTERVAL for values of N from 1
          until it is equal to ROUNDED_SIZE.  */
 
-      emit_insn (ix86_gen_probe_stack_range (sr.reg, sr.reg, GEN_INT (-last)));
+      emit_insn
+	(gen_probe_stack_range (Pmode, sr.reg, sr.reg, GEN_INT (-last)));
 
 
       /* Step 4: probe at FIRST + SIZE if we cannot assert at compile-time
@@ -8182,7 +8168,6 @@ ix86_expand_prologue (void)
     {
       rtx eax = gen_rtx_REG (Pmode, AX_REG);
       rtx r10 = NULL;
-      rtx (*adjust_stack_insn)(rtx, rtx, rtx);
       const bool sp_is_cfa_reg = (m->fs.cfa_reg == stack_pointer_rtx);
       bool eax_live = ix86_eax_live_at_start_p ();
       bool r10_live = false;
@@ -8203,7 +8188,8 @@ ix86_expand_prologue (void)
 	      RTX_FRAME_RELATED_P (insn) = 1;
 	      add_reg_note (insn, REG_FRAME_RELATED_EXPR,
 			    gen_rtx_SET (stack_pointer_rtx,
-					 plus_constant (Pmode, stack_pointer_rtx,
+					 plus_constant (Pmode,
+							stack_pointer_rtx,
 							-UNITS_PER_WORD)));
 	    }
 	}
@@ -8220,21 +8206,18 @@ ix86_expand_prologue (void)
 	      RTX_FRAME_RELATED_P (insn) = 1;
 	      add_reg_note (insn, REG_FRAME_RELATED_EXPR,
 			    gen_rtx_SET (stack_pointer_rtx,
-					 plus_constant (Pmode, stack_pointer_rtx,
+					 plus_constant (Pmode,
+							stack_pointer_rtx,
 							-UNITS_PER_WORD)));
 	    }
 	}
 
       emit_move_insn (eax, GEN_INT (allocate));
-      emit_insn (ix86_gen_allocate_stack_worker (eax, eax));
+      emit_insn (gen_allocate_stack_worker_probe (Pmode, eax, eax));
 
       /* Use the fact that AX still contains ALLOCATE.  */
-      adjust_stack_insn = (Pmode == DImode
-			   ? gen_pro_epilogue_adjust_stack_di_sub
-			   : gen_pro_epilogue_adjust_stack_si_sub);
-
-      insn = emit_insn (adjust_stack_insn (stack_pointer_rtx,
-					   stack_pointer_rtx, eax));
+      insn = emit_insn (gen_pro_epilogue_adjust_stack_sub
+			(Pmode, stack_pointer_rtx, stack_pointer_rtx, eax));
 
       if (sp_is_cfa_reg || TARGET_SEH)
 	{
@@ -8412,8 +8395,9 @@ static void
 ix86_emit_leave (rtx_insn *insn)
 {
   struct machine_function *m = cfun->machine;
+
   if (!insn)
-    insn = emit_insn (ix86_gen_leave ());
+    insn = emit_insn (gen_leave (word_mode));
 
   ix86_add_queued_cfa_restore_notes (insn);
 
@@ -10827,7 +10811,7 @@ legitimize_tls_address (rtx x, enum tls_model model, bool for_mov)
 
 	      start_sequence ();
 	      emit_call_insn
-		(ix86_gen_tls_global_dynamic_64 (rax, x, caddr));
+		(gen_tls_global_dynamic_64 (Pmode, rax, x, caddr));
 	      insns = get_insns ();
 	      end_sequence ();
 
@@ -10881,7 +10865,7 @@ legitimize_tls_address (rtx x, enum tls_model model, bool for_mov)
 
 	      start_sequence ();
 	      emit_call_insn
-		(ix86_gen_tls_local_dynamic_base_64 (rax, caddr));
+		(gen_tls_local_dynamic_base_64 (Pmode, rax, caddr));
 	      insns = get_insns ();
 	      end_sequence ();
 
diff --git a/gcc/config/i386/i386.md b/gcc/config/i386/i386.md
index 3fd11e2..b7328e0 100644
--- a/gcc/config/i386/i386.md
+++ b/gcc/config/i386/i386.md
@@ -13097,7 +13097,15 @@
   [(const_int 0)]
   "ix86_expand_epilogue (2); DONE;")
 
-(define_insn "leave"
+(define_expand "@leave_<mode>"
+  [(parallel
+    [(set (reg:W SP_REG) (plus:W (reg:W BP_REG) (match_dup 0)))
+     (set (reg:W BP_REG) (mem:W (reg:W BP_REG)))
+     (clobber (mem:BLK (scratch)))])]
+  ""
+  "operands[0] = GEN_INT (<MODE_SIZE>);")
+
+(define_insn "*leave"
   [(set (reg:SI SP_REG) (plus:SI (reg:SI BP_REG) (const_int 4)))
    (set (reg:SI BP_REG) (mem:SI (reg:SI BP_REG)))
    (clobber (mem:BLK (scratch)))]
@@ -13105,7 +13113,7 @@
   "leave"
   [(set_attr "type" "leave")])
 
-(define_insn "leave_rex64"
+(define_insn "*leave_rex64"
   [(set (reg:DI SP_REG) (plus:DI (reg:DI BP_REG) (const_int 8)))
    (set (reg:DI BP_REG) (mem:DI (reg:DI BP_REG)))
    (clobber (mem:BLK (scratch)))]
@@ -14274,7 +14282,7 @@
   [(set_attr "type" "multi")
    (set_attr "length" "22")])
 
-(define_expand "tls_global_dynamic_64_<mode>"
+(define_expand "@tls_global_dynamic_64_<mode>"
   [(parallel
     [(set (match_operand:P 0 "register_operand")
 	  (call:P
@@ -14368,7 +14376,7 @@
   [(set_attr "type" "multi")
    (set_attr "length" "22")])
 
-(define_expand "tls_local_dynamic_base_64_<mode>"
+(define_expand "@tls_local_dynamic_base_64_<mode>"
   [(parallel
      [(set (match_operand:P 0 "register_operand")
 	   (call:P
@@ -17830,7 +17838,7 @@
 ;;
 ;; in proper program order.
 
-(define_insn "pro_epilogue_adjust_stack_<mode>_add"
+(define_insn "@pro_epilogue_adjust_stack_add_<mode>"
   [(set (match_operand:P 0 "register_operand" "=r,r")
 	(plus:P (match_operand:P 1 "register_operand" "0,r")
 	        (match_operand:P 2 "<nonmemory_operand>" "r<i>,l<i>")))
@@ -17873,7 +17881,7 @@
 	      (const_string "*")))
    (set_attr "mode" "<MODE>")])
 
-(define_insn "pro_epilogue_adjust_stack_<mode>_sub"
+(define_insn "@pro_epilogue_adjust_stack_sub_<mode>"
   [(set (match_operand:P 0 "register_operand" "=r")
 	(minus:P (match_operand:P 1 "register_operand" "0")
 		 (match_operand:P 2 "register_operand" "r")))
@@ -17884,7 +17892,7 @@
   [(set_attr "type" "alu")
    (set_attr "mode" "<MODE>")])
 
-(define_insn "allocate_stack_worker_probe_<mode>"
+(define_insn "@allocate_stack_worker_probe_<mode>"
   [(set (match_operand:P 0 "register_operand" "=a")
 	(unspec_volatile:P [(match_operand:P 1 "register_operand" "0")]
 			    UNSPECV_STACK_PROBE))
@@ -17910,15 +17918,9 @@
     x = operands[1];
   else
     {
-      rtx (*insn) (rtx, rtx);
-
       x = copy_to_mode_reg (Pmode, operands[1]);
 
-      insn = (TARGET_64BIT
-	      ? gen_allocate_stack_worker_probe_di
-	      : gen_allocate_stack_worker_probe_si);
-
-      emit_insn (insn (x, x));
+      emit_insn (gen_allocate_stack_worker_probe (Pmode, x, x));
     }
 
   x = expand_simple_binop (Pmode, MINUS, stack_pointer_rtx, x,
@@ -17935,16 +17937,13 @@
   [(match_operand 0 "memory_operand")]
   ""
 {
-  rtx (*insn) (rtx, rtx)
-    = (GET_MODE (operands[0]) == DImode
-       ? gen_probe_stack_di : gen_probe_stack_si);
-
-  emit_insn (insn (operands[0], const0_rtx));
+  emit_insn (gen_probe_stack_1
+	     (word_mode, operands[0], const0_rtx));
   DONE;
 })
 
 ;; Use OR for stack probes, this is shorter.
-(define_insn "probe_stack_<mode>"
+(define_insn "@probe_stack_1_<mode>"
   [(set (match_operand:W 0 "memory_operand" "=m")
 	(unspec:W [(match_operand:W 1 "const0_operand")]
 		  UNSPEC_PROBE_STACK))
@@ -17955,7 +17954,7 @@
    (set_attr "mode" "<MODE>")
    (set_attr "length_immediate" "1")])
   
-(define_insn "adjust_stack_and_probe<mode>"
+(define_insn "@adjust_stack_and_probe_<mode>"
   [(set (match_operand:P 0 "register_operand" "=r")
 	(unspec_volatile:P [(match_operand:P 1 "register_operand" "0")]
 			    UNSPECV_PROBE_STACK_RANGE))
@@ -17967,7 +17966,7 @@
   "* return output_adjust_stack_and_probe (operands[0]);"
   [(set_attr "type" "multi")])
 
-(define_insn "probe_stack_range<mode>"
+(define_insn "@probe_stack_range_<mode>"
   [(set (match_operand:P 0 "register_operand" "=r")
 	(unspec_volatile:P [(match_operand:P 1 "register_operand" "0")
 			    (match_operand:P 2 "const_int_operand" "n")]
@@ -19473,17 +19472,12 @@
    (match_operand 1 "memory_operand")]
   ""
 {
-  rtx (*insn)(rtx, rtx);
-
-  insn = (TARGET_LP64
-	  ? gen_stack_protect_set_di
-	  : gen_stack_protect_set_si);
-
-  emit_insn (insn (operands[0], operands[1]));
+  emit_insn (gen_stack_protect_set_1
+	     (ptr_mode, operands[0], operands[1]));
   DONE;
 })
 
-(define_insn "stack_protect_set_<mode>"
+(define_insn "@stack_protect_set_1_<mode>"
   [(set (match_operand:PTR 0 "memory_operand" "=m")
 	(unspec:PTR [(match_operand:PTR 1 "memory_operand" "m")]
 		    UNSPEC_SP_SET))
@@ -19501,20 +19495,15 @@
 {
   rtx flags = gen_rtx_REG (CCZmode, FLAGS_REG);
 
-  rtx (*insn)(rtx, rtx, rtx);
-
-  insn = (TARGET_LP64
-	  ? gen_stack_protect_test_di
-	  : gen_stack_protect_test_si);
-
-  emit_insn (insn (flags, operands[0], operands[1]));
+  emit_insn (gen_stack_protect_test_1
+	     (ptr_mode, flags, operands[0], operands[1]));
 
   emit_jump_insn (gen_cbranchcc4 (gen_rtx_EQ (VOIDmode, flags, const0_rtx),
 				  flags, const0_rtx, operands[2]));
   DONE;
 })
 
-(define_insn "stack_protect_test_<mode>"
+(define_insn "@stack_protect_test_1_<mode>"
   [(set (match_operand:CCZ 0 "flags_reg_operand")
 	(unspec:CCZ [(match_operand:PTR 1 "memory_operand" "m")
 		     (match_operand:PTR 2 "memory_operand" "m")]
@@ -20227,7 +20216,7 @@
   "mwaitx"
   [(set_attr "length" "3")])
 
-(define_insn "monitorx_<mode>"
+(define_insn "@monitorx_<mode>"
   [(unspec_volatile [(match_operand:P 0 "register_operand" "a")
 		     (match_operand:SI 1 "register_operand" "c")
 		     (match_operand:SI 2 "register_operand" "d")]
@@ -20241,7 +20230,7 @@
      (symbol_ref ("(Pmode != word_mode) + 3")))])
 
 ;; CLZERO
-(define_insn "clzero_<mode>"
+(define_insn "@clzero_<mode>"
   [(unspec_volatile [(match_operand: P 0 "register_operand" "a")]
                    UNSPECV_CLZERO)]
   "TARGET_CLZERO"
diff --git a/gcc/config/i386/sse.md b/gcc/config/i386/sse.md
index 7d48402..f053414 100644
--- a/gcc/config/i386/sse.md
+++ b/gcc/config/i386/sse.md
@@ -15568,7 +15568,7 @@
   "mwait"
   [(set_attr "length" "3")])
 
-(define_insn "sse3_monitor_<mode>"
+(define_insn "@sse3_monitor_<mode>"
   [(unspec_volatile [(match_operand:P 0 "register_operand" "a")
 		     (match_operand:SI 1 "register_operand" "c")
 		     (match_operand:SI 2 "register_operand" "d")]
diff --git a/gcc/config/i386/x86-tune-sched.c b/gcc/config/i386/x86-tune-sched.c
index eb84c61..f88e86c 100644
--- a/gcc/config/i386/x86-tune-sched.c
+++ b/gcc/config/i386/x86-tune-sched.c
@@ -27,10 +27,11 @@ along with GCC; see the file COPYING3.  If not see
 #include "tree.h"
 #include "cfghooks.h"
 #include "tm_p.h"
+#include "target.h"
 #include "insn-config.h"
 #include "insn-attr.h"
+#include "insn-opinit.h"
 #include "recog.h"
-#include "target.h"
 
 /* Return the maximum number of instructions a cpu can issue.  */
 
@@ -541,8 +542,7 @@ ix86_macro_fusion_pair_p (rtx_insn *condgen, rtx_insn *condjmp)
 
   condgen_type = get_attr_type (condgen);
   if (condgen_type == TYPE_MULTI
-      && (INSN_CODE (condgen) == CODE_FOR_stack_protect_test_di
-	  || INSN_CODE (condgen) == CODE_FOR_stack_protect_test_si)
+      && INSN_CODE (condgen) == code_for_stack_protect_test_1 (ptr_mode)
       && TARGET_FUSE_ALU_AND_BRANCH)
     {
       /* stack_protect_test_<mode> ends with a sub, which subtracts