s390.md (prologue, [...]): New.

	* s390.md (prologue, epilogue, *return_si, *return_di): New.
	s390.c (find_unused_clobbered_reg, s390_frame_info,
	save_fpr, restore_fpr, s390_emit_prologue, s390_emit_epilogue): New.
	s390-protos.h (s390_emit_prologue, s390_emit_epilogue): Declare.
	s390.c (s390_arg_frame_offset): Use s390_frame_info.
	(leaf_function_flag, cur_is_leaf_function,
	save_fprs, restore_fprs, force_const_mem_late): Remove.
	(s390_function_prologue, s390_function_epilogue): Mostly remove.
	s390.md (lit): New.  Uses ...
	s390.c (s390_output_constant_pool): ... this, so adapt and make global.
	s390-protos.h (s390_output_constant_pool): Declare.
	s390.md (load_multiple, store_multiple): Allow use after reload.
	s390.c (load_multiple_operation, store_multiple_operation): Likewise.
	s390.h (INCOMING_FRAME_SP_OFFSET): Define.
	s390.h (CALL_REALLY_USED_REGISTERS): Define.
	config/s390/linux64.h (CALL_USED_REGISTERS): Remove, now handled ...
	s390.h (CONDITIONAL_REGISTER_USAGE): ... here.
	s390.c (s390_sr_alias_set): New global variable, initialized ...
	(override_options): ... here.  New.
	s390-protos.h (override_options): Declare.
	s390.h (OVERRIDE_OPTIONS): Call it.
	s390.c (s390_function_profiler): New.
	s390-protos.h (s390_function_profiler): Declare.
	s390.h (FUNCTION_PROFILER): Call it.
	s390.c (s390_profile): Remove.

	* s390.c (reg_used_in_mem_p): PC reload counts as memory access.
	(addr_generation_dependency_p): Consider literal pool register loads.
	(s390_adjust_priority): Do not schedule load_multiple.
	s390.md (attribute "type"): Define some additional types.
	(function_unit "integer"): Adapt.
	(many insns): Adapt "type" attribute setting.

	* s390.c (general_s_operand, s_imm_operand): New.
	(s_operand): Remove old definition, call general_s_operand instead.
	s390-protos.h (s_imm_operand): Declare.
	s390.c (base_n_index_p, r_or_s_operand, r_or_s_or_im8_operand,
	r_or_x_or_im16_operand, r_or_im8_operand): Remove.
	s390-protos.h (r_or_s_operand, r_or_s_or_im8_operand,
	r_or_x_or_im16_operand, r_or_im8_operand): Likewise.
	s390.h (PREDICATE_CODES): Add s_imm_operand, remove r_or_s*_operand.
	s390.md (many insns): Rework insn predicates.

	* s390.c (legitimate_pic_operand_p, legitimate_constant_p): Accept all
	non-symbolic constants.  Reload will force them because of ...
	(s390_preferred_reload_class): ... this.  New.
	s390-protos.h (s390_preferred_reload_class): Declare.
	s390.h (PREFERRED_RELOAD_CLASS): Call it.
	s390.md (movdi, movsi, movdf, movsf, *reload_la_64 splitters,
	*reload_la_31 splitters): Handle constants after reload.
	(many insns): no longer force all constants immediately.
	s390.c (legitimate_reload_constant_p): New helper routine.
	s390-protos.h (legitimate_reload_constant_p): Declare.
	s390.c (print_operand): Clean up CONST_INT case, add CONST_DOUBLE case.

	* s390.h (FIRST_PSEUDO_REGISTER, FRAME_POINTER_REGNUM,
	HARD_FRAME_POINTER_REGNUM, REGISTER_NAMES): Add virtual frame pointer.
	(CALL_USED_REGISTERS, CALL_REALLY_USED_REGISTERS): Update.
	(ELIMINABLE_REGS, INITIAL_ELIMINATION_OFFSET): Likewise.
	(REGNO_OK_FOR_INDEX_P, REG_OK_FOR_INDEX_NONSTRICT_P): Likewise.
	(DWARF_FRAME_REGISTERS): Define.
	s390.c (regclass_map): Add virtual frame pointer.
	(legitimate_la_operand_p): Allow use of virtual frame pointer.
	s390.md (*la_ccclobber, *addaddr_ccclobber): New.
	(addaddr, addsi_64): Delete.

	* s390.h (HARD_REGNO_MODE_OK): Allow SImode and DImode values in
	floating point registers.
	(CLASS_CANNOT_CHANGE_MODE, CLASS_CANNOT_CHANGE_MODE_P): Define.
	(ADDR_FP_REGS, GENERAL_FP_REGS): New register classes.
	(REG_CLASS_NAMES, REG_CLASS_CONTENTS): Update.

	* s390.md (movti): Replace multi-insn output with splitters.
	(movdi_31, movdf_31): Likewise.
	(movti_ss, movdi_ss, movsi_ss, movdf_ss, movsf_ss): New.
	(movdi_lhi, movdi_lli, movdi_larl, movsi_lhi, movsi_lli): New.
	(movdi_64, movdi_31, movsi, movdf_64, movdf_31): Adapt.
	(movdf_soft_64, movdf_soft_31, movsf_soft): Remove.
	(movsf_64, movsf_31): Remove, replace by ...
	(movsf): ... this.
	(movqi_64): Use lhi instead of llill.
	(*movstrictqi, *movstricthi): Don't use (strict_low_part (mem)).

	* s390.md (extendsidi2, *extendsidi2, extendhidi2, *extendhidi2,
	extendqidi2, extendhisi2, *extendhisi2, extendqisi2,
	extendqihi2 and associated splitters): Reworked.
	(zero_extendsidi2, zero_extendsidi2, *zero_extendsidi2,
	zero_extendhidi2, *zero_extendhidi2, zero_extendqidi2,
	zero_extendqidi2, *zero_extendqidi2, zero_extendhisi2,
	*zero_extendhisi2_64, zero_extendhisi2_31, zero_extendqisi2,
	*zero_extendqisi2_64, *zero_extendqisi2_mem_31,
	zero_extendqisi2_reg_31, zero_extendqihi2, *zero_extendqisi2_64,
	zero_extendqihi2, zero_extendqihi2_64, zero_extendqihi2_31,
	and associated splitters): Likewise.
	(*sethighqisi, *sethighhisi, *sethighqidi_64, *sethighqidi_31
	and associated splitters): New.
	(truncdisi2, truncdihi2, truncdiqi2, truncsihi2, do_truncsihi2,
	*truncsihi2_64, truncsiqi2, trunchiqi2): Remove.
	(ashlhi3, ashrhi3, lshrhi3, abshi3): Remove.
	s390.h (PROMOTE_PROTOTYPES): Remove.
	config/s390/linux64.h (PROMOTE_PROTOTYPES): Likewise.

	* s390.md (muldi3): Delete, use instead ...
	(mulsidi3): ... this.
	(*muldi3_64): Rename to muldi3.
	(mulsi_6432): Fix template.
	(divdi3, moddi3): Delete, replace by ...
	(divmoddi4): ... this.
	(divmodtidi3): Fix template.
	(divmodtisi3): New.
	(udivdi3, umoddi3): Delete, replace by ...
	(udivmoddi4): ... this.
	(udivmodtidi3): Fix template.
	(divsi3, modsi3): Delete, replace by ...
	(divmodsi4): ... this.
	(divmoddisi3): Fix template.
	(udivsi3, umodsi3): Adapt.

	* s390.md (anddi3): Remove SS alternative, use instead ...
	(anddi3_ss, anddi3_ss_inv): ... these.
	(anddi3_ni): New.
	(andsi3*, andhi3*, andqi3*): Likewise.
	(iordi3): Remove SS alternative, use instead ...
	(iordi3_ss, iordi3_ss_inv): ... these.
	(iordi3_oi): New.
	(iorsi3*, iorhi3*, iorqi3*): Likewise.
	(iordi3_cc, iordi3_cconly, iorsi3_cc, iorsi3_cconly): New.
	(xordi3): Remove SS alternative, use instead ...
	(xordi3_ss, xordi3_ss_inv): ... these.
	(xordi3_oi): New.
	(xorsi3*, xorhi3*, xorqi3*): Likewise.
	(xordi3_cc, xordi3_cconly, xorsi3_cc, xorsi3_cconly): New.
	(one_cmpldi2, one_cmplsi2, one_cmplhi2, one_cmplqi2):
	Expand to XOR with -1.
	(*one_cmpldi2, *one_cmplsi2, *one_cmplhi2): Remove.
	(cmpdi_tm): Delete, replace by ...
	(cmpdi_tm_reg, cmpdi_tm_mem): ... these.
	(cmpsi_cct): Delete, replace by ...
	(cmpsi_tm_reg, cmpsi_tm_mem): ... these.
	(cmpdi_tm2, cmpsi_tm2): Improve.
	(cmphi_tm_sub, cmpqi_tm_sub, cmpqi_tm2, cmpqi_tm): New.
	s390.c (s390_single_hi, s390_extract_hi,
	s390_single_qi, s390_extract_qi): New helper routines.
	s390-protos.h (s390_single_hi, s390_extract_hi,
	s390_single_qi, s390_extract_qi): Declare.
	s390.c (tmxx_operand, const1_operand): Remove.
	s390-protos.h (tmxx_operand, const1_operand): Likewise.
	s390.h (PREDICATE_CODES): Likewise.

	* s390.md (sqrtdf2, sqrtsf2): New.

	* s390.h (CRT_CALL_STATIC_FUNCTION): Define.
	(check_and_change_labels): Remove section-change special case.

	* s390.h (RETURN_ADDR_RTX): Fix use of __builtin_return_address
	in leaf functions.  Needs ...
	(DYNAMIC_CHAIN_RTX):  ... this.  New.

	* s390.c (emit_pic_move): Don't generate pseudos if no_new_pseudos.

	* s390.md (movstrdix_64, movstrsix_31, movstrdi_64, movstrsi_31,
	clrstrsi_64, clrstrsi_31, cmpstr_64, cmpstr_31): Do not clobber
	input operands using a match_dup clause.
	(movstrdi, movstrsi, clrstrdi, clrstrsi, cmpstrdi, cmpstrsi): Adapt.

	* s390.md (floatdidf2, floatdisf2, floatsidf2, floatsidf2_ieee,
	floatsisf2, floatsisf2_ieee): Add missing CC clobber.

	* s390.md (floatsidf2_ibm): Use correct operand.

	* s390.md (fixuns_truncdfdi2, fixuns_truncdfsi2, fix_truncdfsi2,
	fixuns_truncsfdi2, fixuns_truncsfsi2, floatsidf2): Remove use of
	non-portable constants.
	s390.c (s390_gen_rtx_const_DI): New helper routine.
	s390-protos.h (s390_gen_rtx_const_DI): Declare.

	* s390.h (ASM_OUTPUT_SPECIAL_POOL_ENTRY): Fix alignment.

	* config/s390/linux.h (ASM_OUTPUT_SHORT, ASM_OUTPUT_CHAR,
	ASM_OUTPUT_BYTE): Clean up assembly output.
	(ASM_OUTPUT_SKIP, ASM_OUTPUT_ALIGN): Remove duplicate definitions.
	(ASM_OUTPUT_ASCII): Remove.

	* config/s390/t-linux (CRTSTUFF_T_CFLAGS_S): Define.

From-SVN: r48058

8 files changed, 3525 insertions, 2296 deletions

diff --git a/gcc/ChangeLog b/gcc/ChangeLog
index 47570f9..7eba009 100644
--- a/gcc/ChangeLog
+++ b/gcc/ChangeLog
@@ -1,3 +1,190 @@
+2001-12-15  Ulrich Weigand  <uweigand@de.ibm.com>
+
+	* s390.md (prologue, epilogue, *return_si, *return_di): New.
+	s390.c (find_unused_clobbered_reg, s390_frame_info, 
+	save_fpr, restore_fpr, s390_emit_prologue, s390_emit_epilogue): New.
+	s390-protos.h (s390_emit_prologue, s390_emit_epilogue): Declare.
+	s390.c (s390_arg_frame_offset): Use s390_frame_info.
+	(leaf_function_flag, cur_is_leaf_function,
+	save_fprs, restore_fprs, force_const_mem_late): Remove.
+	(s390_function_prologue, s390_function_epilogue): Mostly remove.
+	s390.md (lit): New.  Uses ...
+	s390.c (s390_output_constant_pool): ... this, so adapt and make global.
+	s390-protos.h (s390_output_constant_pool): Declare.
+	s390.md (load_multiple, store_multiple): Allow use after reload.
+	s390.c (load_multiple_operation, store_multiple_operation): Likewise.
+	s390.h (INCOMING_FRAME_SP_OFFSET): Define.
+	s390.h (CALL_REALLY_USED_REGISTERS): Define.
+	config/s390/linux64.h (CALL_USED_REGISTERS): Remove, now handled ...
+	s390.h (CONDITIONAL_REGISTER_USAGE): ... here.
+	s390.c (s390_sr_alias_set): New global variable, initialized ...
+	(override_options): ... here.  New.
+	s390-protos.h (override_options): Declare.
+	s390.h (OVERRIDE_OPTIONS): Call it.
+	s390.c (s390_function_profiler): New.
+	s390-protos.h (s390_function_profiler): Declare.
+	s390.h (FUNCTION_PROFILER): Call it.
+	s390.c (s390_profile): Remove.
+
+	* s390.c (reg_used_in_mem_p): PC reload counts as memory access.
+	(addr_generation_dependency_p): Consider literal pool register loads.
+	(s390_adjust_priority): Do not schedule load_multiple.
+	s390.md (attribute "type"): Define some additional types.
+	(function_unit "integer"): Adapt.
+	(many insns): Adapt "type" attribute setting.
+
+	* s390.c (general_s_operand, s_imm_operand): New.
+	(s_operand): Remove old definition, call general_s_operand instead.
+	s390-protos.h (s_imm_operand): Declare.
+	s390.c (base_n_index_p, r_or_s_operand, r_or_s_or_im8_operand, 
+	r_or_x_or_im16_operand, r_or_im8_operand): Remove.
+	s390-protos.h (r_or_s_operand, r_or_s_or_im8_operand,
+	r_or_x_or_im16_operand, r_or_im8_operand): Likewise.
+	s390.h (PREDICATE_CODES): Add s_imm_operand, remove r_or_s*_operand.
+	s390.md (many insns): Rework insn predicates.
+
+	* s390.c (legitimate_pic_operand_p, legitimate_constant_p): Accept all
+	non-symbolic constants.  Reload will force them because of ...
+	(s390_preferred_reload_class): ... this.  New.
+	s390-protos.h (s390_preferred_reload_class): Declare.
+	s390.h (PREFERRED_RELOAD_CLASS): Call it.
+	s390.md (movdi, movsi, movdf, movsf, *reload_la_64 splitters,
+	*reload_la_31 splitters): Handle constants after reload.
+	(many insns): no longer force all constants immediately.
+	s390.c (legitimate_reload_constant_p): New helper routine.
+	s390-protos.h (legitimate_reload_constant_p): Declare.
+	s390.c (print_operand): Clean up CONST_INT case, add CONST_DOUBLE case.
+
+	* s390.h (FIRST_PSEUDO_REGISTER, FRAME_POINTER_REGNUM, 
+	HARD_FRAME_POINTER_REGNUM, REGISTER_NAMES): Add virtual frame pointer.
+	(CALL_USED_REGISTERS, CALL_REALLY_USED_REGISTERS): Update.
+	(ELIMINABLE_REGS, INITIAL_ELIMINATION_OFFSET): Likewise.
+	(REGNO_OK_FOR_INDEX_P, REG_OK_FOR_INDEX_NONSTRICT_P): Likewise.
+	(DWARF_FRAME_REGISTERS): Define.
+	s390.c (regclass_map): Add virtual frame pointer.
+	(legitimate_la_operand_p): Allow use of virtual frame pointer.
+	s390.md (*la_ccclobber, *addaddr_ccclobber): New.
+	(addaddr, addsi_64): Delete.
+
+	* s390.h (HARD_REGNO_MODE_OK): Allow SImode and DImode values in 
+	floating point registers.
+	(CLASS_CANNOT_CHANGE_MODE, CLASS_CANNOT_CHANGE_MODE_P): Define.
+	(ADDR_FP_REGS, GENERAL_FP_REGS): New register classes.
+	(REG_CLASS_NAMES, REG_CLASS_CONTENTS): Update.
+
+	* s390.md (movti): Replace multi-insn output with splitters.
+	(movdi_31, movdf_31): Likewise.
+	(movti_ss, movdi_ss, movsi_ss, movdf_ss, movsf_ss): New.
+	(movdi_lhi, movdi_lli, movdi_larl, movsi_lhi, movsi_lli): New.
+	(movdi_64, movdi_31, movsi, movdf_64, movdf_31): Adapt.
+	(movdf_soft_64, movdf_soft_31, movsf_soft): Remove.
+	(movsf_64, movsf_31): Remove, replace by ...
+	(movsf): ... this.
+	(movqi_64): Use lhi instead of llill.
+	(*movstrictqi, *movstricthi): Don't use (strict_low_part (mem)).
+
+	* s390.md (extendsidi2, *extendsidi2, extendhidi2, *extendhidi2,
+	extendqidi2, extendhisi2, *extendhisi2, extendqisi2,
+	extendqihi2 and associated splitters): Reworked.
+	(zero_extendsidi2, zero_extendsidi2, *zero_extendsidi2,
+	zero_extendhidi2, *zero_extendhidi2, zero_extendqidi2,
+	zero_extendqidi2, *zero_extendqidi2, zero_extendhisi2,
+	*zero_extendhisi2_64, zero_extendhisi2_31, zero_extendqisi2,
+	*zero_extendqisi2_64, *zero_extendqisi2_mem_31,
+	zero_extendqisi2_reg_31, zero_extendqihi2, *zero_extendqisi2_64,
+	zero_extendqihi2, zero_extendqihi2_64, zero_extendqihi2_31,
+	and associated splitters): Likewise.
+	(*sethighqisi, *sethighhisi, *sethighqidi_64, *sethighqidi_31 
+	and associated splitters): New.
+	(truncdisi2, truncdihi2, truncdiqi2, truncsihi2, do_truncsihi2, 
+	*truncsihi2_64, truncsiqi2, trunchiqi2): Remove.
+	(ashlhi3, ashrhi3, lshrhi3, abshi3): Remove.
+	s390.h (PROMOTE_PROTOTYPES): Remove.
+	config/s390/linux64.h (PROMOTE_PROTOTYPES): Likewise.
+
+	* s390.md (muldi3): Delete, use instead ...
+	(mulsidi3): ... this.
+	(*muldi3_64): Rename to muldi3.
+	(mulsi_6432): Fix template.
+	(divdi3, moddi3): Delete, replace by ...
+	(divmoddi4): ... this.
+	(divmodtidi3): Fix template.
+	(divmodtisi3): New.
+	(udivdi3, umoddi3): Delete, replace by ...
+	(udivmoddi4): ... this.
+	(udivmodtidi3): Fix template.
+	(divsi3, modsi3): Delete, replace by ...
+	(divmodsi4): ... this.
+	(divmoddisi3): Fix template.
+	(udivsi3, umodsi3): Adapt.
+
+	* s390.md (anddi3): Remove SS alternative, use instead ...
+	(anddi3_ss, anddi3_ss_inv): ... these.
+	(anddi3_ni): New.
+	(andsi3*, andhi3*, andqi3*): Likewise.
+	(iordi3): Remove SS alternative, use instead ...
+	(iordi3_ss, iordi3_ss_inv): ... these.
+	(iordi3_oi): New.
+	(iorsi3*, iorhi3*, iorqi3*): Likewise.
+	(iordi3_cc, iordi3_cconly, iorsi3_cc, iorsi3_cconly): New.
+	(xordi3): Remove SS alternative, use instead ...
+	(xordi3_ss, xordi3_ss_inv): ... these.
+	(xordi3_oi): New.
+	(xorsi3*, xorhi3*, xorqi3*): Likewise.
+	(xordi3_cc, xordi3_cconly, xorsi3_cc, xorsi3_cconly): New.
+	(one_cmpldi2, one_cmplsi2, one_cmplhi2, one_cmplqi2):
+	Expand to XOR with -1.
+	(*one_cmpldi2, *one_cmplsi2, *one_cmplhi2): Remove.
+	(cmpdi_tm): Delete, replace by ...
+	(cmpdi_tm_reg, cmpdi_tm_mem): ... these.
+	(cmpsi_cct): Delete, replace by ...
+	(cmpsi_tm_reg, cmpsi_tm_mem): ... these.
+	(cmpdi_tm2, cmpsi_tm2): Improve.
+	(cmphi_tm_sub, cmpqi_tm_sub, cmpqi_tm2, cmpqi_tm): New.
+	s390.c (s390_single_hi, s390_extract_hi, 
+	s390_single_qi, s390_extract_qi): New helper routines.
+	s390-protos.h (s390_single_hi, s390_extract_hi, 
+	s390_single_qi, s390_extract_qi): Declare.
+	s390.c (tmxx_operand, const1_operand): Remove.
+	s390-protos.h (tmxx_operand, const1_operand): Likewise.
+	s390.h (PREDICATE_CODES): Likewise.
+
+	* s390.md (sqrtdf2, sqrtsf2): New.
+
+	* s390.h (CRT_CALL_STATIC_FUNCTION): Define.
+	(check_and_change_labels): Remove section-change special case.
+
+	* s390.h (RETURN_ADDR_RTX): Fix use of __builtin_return_address 
+	in leaf functions.  Needs ...
+	(DYNAMIC_CHAIN_RTX):  ... this.  New.
+
+	* s390.c (emit_pic_move): Don't generate pseudos if no_new_pseudos.
+
+	* s390.md (movstrdix_64, movstrsix_31, movstrdi_64, movstrsi_31,
+	clrstrsi_64, clrstrsi_31, cmpstr_64, cmpstr_31): Do not clobber 
+	input operands using a match_dup clause.
+	(movstrdi, movstrsi, clrstrdi, clrstrsi, cmpstrdi, cmpstrsi): Adapt.
+
+	* s390.md (floatdidf2, floatdisf2, floatsidf2, floatsidf2_ieee,
+	floatsisf2, floatsisf2_ieee): Add missing CC clobber.
+
+	* s390.md (floatsidf2_ibm): Use correct operand.
+
+	* s390.md (fixuns_truncdfdi2, fixuns_truncdfsi2, fix_truncdfsi2,
+	fixuns_truncsfdi2, fixuns_truncsfsi2, floatsidf2): Remove use of
+	non-portable constants.
+	s390.c (s390_gen_rtx_const_DI): New helper routine.
+	s390-protos.h (s390_gen_rtx_const_DI): Declare.
+
+	* s390.h (ASM_OUTPUT_SPECIAL_POOL_ENTRY): Fix alignment.
+
+	* config/s390/linux.h (ASM_OUTPUT_SHORT, ASM_OUTPUT_CHAR,
+	ASM_OUTPUT_BYTE): Clean up assembly output.
+	(ASM_OUTPUT_SKIP, ASM_OUTPUT_ALIGN): Remove duplicate definitions.
+	(ASM_OUTPUT_ASCII): Remove.
+
+	* config/s390/t-linux (CRTSTUFF_T_CFLAGS_S): Define.
+
 2001-12-15  Zack Weinberg  <zack@codesourcery.com>
 
 	* unwind-dw2-fde-glibc.c: #define _Unwind_Find_FDE to itself
diff --git a/gcc/config/s390/linux.h b/gcc/config/s390/linux.h
index 75a483d..ce1477e 100644
--- a/gcc/config/s390/linux.h
+++ b/gcc/config/s390/linux.h
@@ -154,19 +154,19 @@ do { fprintf (FILE, "%s\t", ASM_LONG);          \
    is this supposed to do align too?? */
 
 #define ASM_OUTPUT_SHORT(FILE, VALUE)           \
-( fprintf (FILE, "%s ", ASM_SHORT),             \
+( fprintf (FILE, "%s\t", ASM_SHORT),            \
   output_addr_const (FILE, (VALUE)),            \
   putc ('\n',FILE))
 
 #define ASM_OUTPUT_CHAR(FILE, VALUE)            \
-( fprintf (FILE, "%s ", ASM_BYTE_OP),           \
+( fprintf (FILE, "\t%s\t", ASM_BYTE_OP),        \
   output_addr_const (FILE, (VALUE)),            \
   putc ('\n', FILE))
 
 /* This is how to output an assembler line for a numeric constant byte.  */
 
 #define ASM_OUTPUT_BYTE(FILE, VALUE)  \
-  fprintf ((FILE), "%s 0x%x\n", ASM_BYTE_OP, (int)(VALUE))
+  fprintf ((FILE), "\t%s\t0x%x\n", ASM_BYTE_OP, (int)(VALUE))
 
      /* internal macro to output long */
 #define _ASM_OUTPUT_LONG(FILE, VALUE)                                   \
@@ -191,75 +191,16 @@ do { fprintf (FILE, "%s\t", ASM_LONG);          \
    that says to advance the location counter
    to a multiple of 2**LOG bytes.  */
 
-#define ASM_OUTPUT_ALIGN(FILE, LOG)      \
-    if ((LOG)!=0) fprintf ((FILE), "\t.align %d\n", 1<<(LOG))
-
-/* This is how to output an assembler line
-   that says to advance the location counter by SIZE bytes.  */
-
-#undef ASM_OUTPUT_SKIP 
-#define ASM_OUTPUT_SKIP(FILE, SIZE)  \
-  fprintf ((FILE), "\t.set .,.+%u\n", (SIZE))
-
-/* This is how to output an assembler line
-   that says to advance the location counter
-   to a multiple of 2**LOG bytes.  */
-
+#undef ASM_OUTPUT_ALIGN
 #define ASM_OUTPUT_ALIGN(FILE, LOG)	\
-    if ((LOG)!=0) fprintf ((FILE), "\t.align %d\n", 1<<(LOG))
+    if ((LOG)!=0) fprintf ((FILE), "\t.align\t%d\n", 1<<(LOG))
 
 /* This is how to output an assembler line
    that says to advance the location counter by SIZE bytes.  */
 
+#undef ASM_OUTPUT_SKIP
 #define ASM_OUTPUT_SKIP(FILE, SIZE)  \
-  fprintf ((FILE), "\t.set .,.+%u\n", (SIZE))
-
-/* The routine used to output sequences of byte values.  We use a special
-   version of this for most svr4 targets because doing so makes the
-   generated assembly code more compact (and thus faster to assemble)
-   as well as more readable.  Note that if we find subparts of the
-   character sequence which end with NUL (and which are shorter than
-   STRING_LIMIT) we output those using ASM_OUTPUT_LIMITED_STRING.  */
-
-#undef ASM_OUTPUT_ASCII
-#define ASM_OUTPUT_ASCII(FILE, STR, LENGTH)                             \
-do {                                                                    \
-      register const unsigned char *_ascii_bytes = (const unsigned char *) (STR);   \
-      register const unsigned char *limit = _ascii_bytes + (LENGTH);          \
-      register unsigned bytes_in_chunk = 0;                             \
-      for (; _ascii_bytes < limit; _ascii_bytes++)                      \
-        {                                                               \
-          register const unsigned char *p;                                    \
-          if (bytes_in_chunk >= 64)                                     \
-            {                                                           \
-              fputc ('\n', (FILE));                                     \
-              bytes_in_chunk = 0;                                       \
-            }                                                           \
-          for (p = _ascii_bytes; p < limit && *p != '\0'; p++)          \
-            continue;                                                   \
-          if (p < limit && (p - _ascii_bytes) <= (long)STRING_LIMIT)    \
-            {                                                           \
-              if (bytes_in_chunk > 0)                                   \
-                {                                                       \
-                  fputc ('\n', (FILE));                                 \
-                  bytes_in_chunk = 0;                                   \
-                }                                                       \
-              ASM_OUTPUT_LIMITED_STRING ((FILE), _ascii_bytes);         \
-              _ascii_bytes = p;                                         \
-            }                                                           \
-          else                                                          \
-            {                                                           \
-              if (bytes_in_chunk == 0)                                  \
-                fprintf ((FILE), "%s\t", ASM_BYTE_OP);                  \
-              else                                                      \
-                fputc (',', (FILE));                                    \
-              fprintf ((FILE), "0x%02x", *_ascii_bytes);                \
-              bytes_in_chunk += 5;                                      \
-            }                                                           \
-        }                                                               \
-      if (bytes_in_chunk > 0)                                           \
-        fprintf ((FILE), "\n");                                         \
-} while (0)
+  fprintf ((FILE), "\t.set\t.,.+%u\n", (SIZE))
 
 /* Output before read-only data.  */
 
diff --git a/gcc/config/s390/linux64.h b/gcc/config/s390/linux64.h
index 0fdd173..a65cf90 100644
--- a/gcc/config/s390/linux64.h
+++ b/gcc/config/s390/linux64.h
@@ -59,21 +59,7 @@ Boston, MA 02111-1307, USA.  */
 	%{static:-static}}}"
 #endif
 
-#undef PROMOTE_PROTOTYPES 
 #undef MASK_RETURN_ADDR 
 #undef SELECT_SECTION
 
-/* With 64 bit new linkage for floating point registers.  */
-#undef CALL_USED_REGISTERS			
-#define CALL_USED_REGISTERS			\
-{ 1, 1, 1, 1, 					\
-  1, 1, 0, 0, 					\
-  0, 0, 0, 0, 					\
-  0, 1, 1, 1,					\
-  1, 1, 1, 1, 					\
-  1, 1, 1, 1, 					\
-  0, 0, 0, 0, 					\
-  0, 0, 0, 0, 					\
-  1, 1 }
-
 #endif
diff --git a/gcc/config/s390/s390-protos.h b/gcc/config/s390/s390-protos.h
index baee5a0..b426dac 100644
--- a/gcc/config/s390/s390-protos.h
+++ b/gcc/config/s390/s390-protos.h
@@ -22,24 +22,27 @@ Boston, MA 02111-1307, USA.  */
 /* Declare functions in s390.c.  */
 
 extern void optimization_options PARAMS ((int, int));
+extern void override_options PARAMS ((void));
 extern int s390_arg_frame_offset PARAMS ((void));
 extern void s390_function_prologue PARAMS ((FILE *, HOST_WIDE_INT));
 extern void s390_function_epilogue PARAMS ((FILE *, HOST_WIDE_INT));
+extern void s390_emit_prologue PARAMS ((void));
+extern void s390_emit_epilogue PARAMS ((void));
+extern void s390_function_profiler PARAMS ((FILE *, int));
 
 #ifdef RTX_CODE
 extern int const0_operand PARAMS ((rtx, enum machine_mode));
-extern int const1_operand PARAMS ((rtx, enum machine_mode));
 extern int larl_operand PARAMS ((rtx, enum machine_mode));
 extern int fp_operand PARAMS ((rtx, enum machine_mode));
 extern int s_operand PARAMS ((rtx, enum machine_mode));
-extern int r_or_s_operand PARAMS ((rtx, enum machine_mode));
-extern int r_or_s_or_im8_operand PARAMS ((rtx, enum machine_mode));
-extern int r_or_x_or_im16_operand PARAMS ((rtx, enum machine_mode));
-extern int r_or_im8_operand PARAMS ((rtx, enum machine_mode));
-extern int tmxx_operand PARAMS ((rtx, enum machine_mode));
+extern int s_imm_operand PARAMS ((rtx, enum machine_mode));
 extern int bras_sym_operand PARAMS ((rtx, enum machine_mode));
 extern int load_multiple_operation PARAMS ((rtx, enum machine_mode));
 extern int store_multiple_operation PARAMS ((rtx, enum machine_mode));
+extern int s390_single_hi PARAMS ((rtx, enum machine_mode, int));
+extern int s390_extract_hi PARAMS ((rtx, enum machine_mode, int));
+extern int s390_single_qi PARAMS ((rtx, enum machine_mode, int));
+extern int s390_extract_qi PARAMS ((rtx, enum machine_mode, int));
 
 extern int s390_match_ccmode PARAMS ((rtx, enum machine_mode));
 extern enum machine_mode s390_select_ccmode PARAMS ((enum rtx_code, rtx, rtx));
@@ -47,18 +50,22 @@ extern int symbolic_reference_mentioned_p PARAMS ((rtx));
 extern int legitimate_la_operand_p PARAMS ((rtx));
 extern int legitimate_pic_operand_p PARAMS ((rtx));
 extern int legitimate_constant_p PARAMS ((rtx));
+extern int legitimate_reload_constant_p PARAMS ((rtx));
 extern int legitimate_address_p PARAMS ((enum machine_mode, rtx, int));
 extern rtx legitimize_pic_address PARAMS ((rtx, rtx));
 extern rtx legitimize_address PARAMS ((rtx, rtx, enum machine_mode));
+extern enum reg_class s390_preferred_reload_class PARAMS ((rtx, enum reg_class));
 extern void emit_pic_move PARAMS ((rtx *, enum machine_mode));
 
 extern void s390_output_symbolic_const PARAMS ((FILE *, rtx));
 extern void print_operand_address PARAMS ((FILE *, rtx));
 extern void print_operand PARAMS ((FILE *, rtx, int));
+extern void s390_output_constant_pool PARAMS ((FILE *));
 extern int s390_stop_dump_lit_p PARAMS ((rtx));
 extern void s390_dump_literal_pool PARAMS ((rtx, rtx));
 extern void s390_trampoline_template PARAMS ((FILE *));
 extern void s390_initialize_trampoline PARAMS ((rtx, rtx, rtx));
+extern rtx s390_gen_rtx_const_DI PARAMS ((int, int));
 #endif /* RTX_CODE */
 
 #ifdef TREE_CODE
diff --git a/gcc/config/s390/s390.c b/gcc/config/s390/s390.c
index 34945b0..743276c 100644
--- a/gcc/config/s390/s390.c
+++ b/gcc/config/s390/s390.c
@@ -39,6 +39,7 @@ Boston, MA 02111-1307, USA.  */
 #include "expr.h"
 #include "toplev.h"
 #include "basic-block.h"
+#include "integrate.h"
 #include "ggc.h"
 #include "target.h"
 #include "target-def.h"
@@ -70,6 +71,9 @@ struct gcc_target targetm = TARGET_INITIALIZER;
 
 extern int reload_completed;
 
+/* The alias set for prologue/epilogue register save/restore.  */
+static int s390_sr_alias_set = 0;
+
 /* Function count for creating unique internal labels in a compile unit.  */
 int  s390_function_count = 0;
 
@@ -93,11 +97,26 @@ struct s390_address
   rtx disp;
 };
 
+/* Structure containing information for prologue and epilogue.  */ 
+
+struct s390_frame
+{
+  int frame_pointer_p;
+  int return_reg_saved_p;
+  int save_fprs_p;
+  int first_save_gpr;
+  int first_restore_gpr;
+  int last_save_gpr;
+  int arg_frame_offset;
+
+  HOST_WIDE_INT frame_size;
+};
+
 static int s390_match_ccmode_set PARAMS ((rtx, enum machine_mode));
 static int s390_branch_condition_mask PARAMS ((rtx));
 static const char *s390_branch_condition_mnemonic PARAMS ((rtx, int));
-static int base_n_index_p PARAMS ((rtx));
 static int check_mode PARAMS ((rtx, enum machine_mode *));
+static int general_s_operand PARAMS ((rtx, enum machine_mode, int));
 static int s390_decompose_address PARAMS ((rtx, struct s390_address *, int));
 static int reg_used_in_mem_p PARAMS ((int, rtx));
 static int addr_generation_dependency_p PARAMS ((rtx, rtx));
@@ -106,12 +125,10 @@ static int far_away PARAMS ((int, int));
 static rtx check_and_change_labels PARAMS ((rtx, int *));
 static void s390_final_chunkify PARAMS ((int));
 static int save_fprs_p PARAMS ((void));
-static int cur_is_leaf_function PARAMS ((void));
-static int save_fprs PARAMS ((FILE *, long, int));
-static int restore_fprs PARAMS ((FILE *, long, int));
-static void s390_output_constant_pool PARAMS ((FILE *));
-static rtx s390_force_const_mem_late PARAMS ((rtx));
-static rtx s390_force_const_mem_symbol PARAMS ((const char *, int, int));
+static int find_unused_clobbered_reg PARAMS ((void));
+static void s390_frame_info PARAMS ((struct s390_frame *));
+static rtx save_fpr PARAMS ((rtx, int, int));
+static rtx restore_fpr PARAMS ((rtx, int, int));
 static int s390_function_arg_size PARAMS ((enum machine_mode, tree));
 
  
@@ -339,6 +356,212 @@ s390_branch_condition_mnemonic (code, inv)
   return mnemonic[mask];
 }
 
+/* If OP is an integer constant of mode MODE with exactly one
+   HImode subpart unequal to DEF, return the number of that 
+   subpart.  As a special case, all HImode subparts of OP are
+   equal to DEF, return zero.  Otherwise, return -1.  */
+
+int
+s390_single_hi (op, mode, def)
+     rtx op;
+     enum machine_mode mode;
+     int def;
+{
+  if (GET_CODE (op) == CONST_INT)
+    {
+      unsigned HOST_WIDE_INT value;
+      int n_parts = GET_MODE_SIZE (mode) / 2;
+      int i, part = -1;
+
+      for (i = 0; i < n_parts; i++)
+        {
+          if (i == 0)
+            value = (unsigned HOST_WIDE_INT) INTVAL (op);
+          else
+            value >>= 16;
+
+          if ((value & 0xffff) != (unsigned)(def & 0xffff))
+            {
+              if (part != -1)
+                return -1;
+              else
+                part = i;
+            }
+        }
+
+      return part == -1 ? 0 : (n_parts - 1 - part);
+    }
+
+  else if (GET_CODE (op) == CONST_DOUBLE
+           && GET_MODE (op) == VOIDmode)
+    {
+      unsigned HOST_WIDE_INT value;
+      int n_parts = GET_MODE_SIZE (mode) / 2;
+      int i, part = -1;
+
+      for (i = 0; i < n_parts; i++)
+        {
+          if (i == 0)
+            value = (unsigned HOST_WIDE_INT) CONST_DOUBLE_LOW (op);
+          else if (i == HOST_BITS_PER_WIDE_INT / 16)
+            value = (unsigned HOST_WIDE_INT) CONST_DOUBLE_HIGH (op);
+          else
+            value >>= 16;
+
+          if ((value & 0xffff) != (unsigned)(def & 0xffff))
+            {
+              if (part != -1)
+                return -1;
+              else
+                part = i;
+            }
+        }
+
+      return part == -1 ? 0 : (n_parts - 1 - part);
+    }
+
+  return -1;      
+}
+
+/* Extract the HImode part number PART from integer 
+   constant OP of mode MODE.  */
+
+int
+s390_extract_hi (op, mode, part)
+    rtx op;
+    enum machine_mode mode;
+    int part;
+{
+  int n_parts = GET_MODE_SIZE (mode) / 2;
+  if (part < 0 || part >= n_parts)
+    abort();
+  else
+    part = n_parts - 1 - part;
+
+  if (GET_CODE (op) == CONST_INT)
+    {
+      unsigned HOST_WIDE_INT value = (unsigned HOST_WIDE_INT) INTVAL (op);
+      return ((value >> (16 * part)) & 0xffff);
+    }
+  else if (GET_CODE (op) == CONST_DOUBLE
+           && GET_MODE (op) == VOIDmode)
+    {
+      unsigned HOST_WIDE_INT value;
+      if (part < HOST_BITS_PER_WIDE_INT / 16)
+        value = (unsigned HOST_WIDE_INT) CONST_DOUBLE_LOW (op);
+      else
+        value = (unsigned HOST_WIDE_INT) CONST_DOUBLE_HIGH (op),
+        part -= HOST_BITS_PER_WIDE_INT / 16;
+
+      return ((value >> (16 * part)) & 0xffff); 
+    }
+
+  abort ();
+}
+
+/* If OP is an integer constant of mode MODE with exactly one
+   QImode subpart unequal to DEF, return the number of that 
+   subpart.  As a special case, all QImode subparts of OP are
+   equal to DEF, return zero.  Otherwise, return -1.  */
+
+int
+s390_single_qi (op, mode, def)
+     rtx op;
+     enum machine_mode mode;
+     int def;
+{
+  if (GET_CODE (op) == CONST_INT)
+    {
+      unsigned HOST_WIDE_INT value;
+      int n_parts = GET_MODE_SIZE (mode);
+      int i, part = -1;
+
+      for (i = 0; i < n_parts; i++)
+        {
+          if (i == 0)
+            value = (unsigned HOST_WIDE_INT) INTVAL (op);
+          else
+            value >>= 8;
+
+          if ((value & 0xff) != (unsigned)(def & 0xff))
+            {
+              if (part != -1)
+                return -1;
+              else
+                part = i;
+            }
+        }
+
+      return part == -1 ? 0 : (n_parts - 1 - part);
+    }
+
+  else if (GET_CODE (op) == CONST_DOUBLE
+           && GET_MODE (op) == VOIDmode)
+    {
+      unsigned HOST_WIDE_INT value;
+      int n_parts = GET_MODE_SIZE (mode);
+      int i, part = -1;
+
+      for (i = 0; i < n_parts; i++)
+        {
+          if (i == 0)
+            value = (unsigned HOST_WIDE_INT) CONST_DOUBLE_LOW (op);
+          else if (i == HOST_BITS_PER_WIDE_INT / 8)
+            value = (unsigned HOST_WIDE_INT) CONST_DOUBLE_HIGH (op);
+          else
+            value >>= 8;
+
+          if ((value & 0xff) != (unsigned)(def & 0xff))
+            {
+              if (part != -1)
+                return -1;
+              else
+                part = i;
+            }
+        }
+
+      return part == -1 ? 0 : (n_parts - 1 - part);
+    }
+
+  return -1;      
+}
+
+/* Extract the QImode part number PART from integer 
+   constant OP of mode MODE.  */
+
+int
+s390_extract_qi (op, mode, part)
+    rtx op;
+    enum machine_mode mode;
+    int part;
+{
+  int n_parts = GET_MODE_SIZE (mode);
+  if (part < 0 || part >= n_parts)
+    abort();
+  else
+    part = n_parts - 1 - part;
+
+  if (GET_CODE (op) == CONST_INT)
+    {
+      unsigned HOST_WIDE_INT value = (unsigned HOST_WIDE_INT) INTVAL (op);
+      return ((value >> (8 * part)) & 0xff);
+    }
+  else if (GET_CODE (op) == CONST_DOUBLE
+           && GET_MODE (op) == VOIDmode)
+    {
+      unsigned HOST_WIDE_INT value;
+      if (part < HOST_BITS_PER_WIDE_INT / 8)
+        value = (unsigned HOST_WIDE_INT) CONST_DOUBLE_LOW (op);
+      else
+        value = (unsigned HOST_WIDE_INT) CONST_DOUBLE_HIGH (op),
+        part -= HOST_BITS_PER_WIDE_INT / 8;
+
+      return ((value >> (8 * part)) & 0xff); 
+    }
+
+  abort ();
+}
+
 
 /* Change optimizations to be performed, depending on the 
    optimization level.
@@ -360,6 +583,13 @@ optimization_options (level, size)
 #endif
 }
 
+void
+override_options ()
+{
+  /* Acquire a unique set number for our register saves and restores.  */
+  s390_sr_alias_set = new_alias_set ();
+}
+
 
 /* Map for smallest class containing reg regno.  */
 
@@ -372,7 +602,7 @@ enum reg_class regclass_map[FIRST_PSEUDO_REGISTER] =
   FP_REGS,      FP_REGS,   FP_REGS,   FP_REGS,
   FP_REGS,      FP_REGS,   FP_REGS,   FP_REGS,
   FP_REGS,      FP_REGS,   FP_REGS,   FP_REGS,
-  ADDR_REGS,    NO_REGS 
+  ADDR_REGS,    NO_REGS,   ADDR_REGS 
 };
 
 
@@ -388,32 +618,6 @@ const0_operand (op, mode)
   return op == CONST0_RTX (mode);
 }
 
-/* Return true if OP a (const_int 1) operand.
-   OP is the current operation.
-   MODE is the current operation mode.  */
- 
-int
-const1_operand (op, mode)
-     register rtx op;
-     enum machine_mode mode;
-{
-  return op == CONST1_RTX (mode);
-}
-
-/* Return true if OP needs base and index register.  */
-
-static int 
-base_n_index_p (op)
-     register rtx op;
-{
-  if ((GET_CODE (op) == PLUS) &&
-      (GET_CODE (XEXP (op, 0)) == PLUS ||
-       GET_CODE (XEXP (op, 1)) == PLUS ||
-       GET_CODE (XEXP (op, 1)) == REG ))
-    return 1;
-  return 0;
-}
-
 /* Return true if the mode of operand OP matches MODE.
    If MODE is set to VOIDmode, set it to the mode of OP.  */ 
 
@@ -503,146 +707,90 @@ fp_operand (op, mode)
     return 0;
 }
 
-/* Return true if OP is a valid S operand for an RS, SI or SS type instruction.
-   OP is the current operation.
-   MODE is the current operation mode.  */
-
-int
-s_operand (op, mode)
-     register rtx op;
-     enum machine_mode mode;
-{
-  register enum rtx_code code = GET_CODE (op);
-
-  if (! check_mode (op,&mode))
-    return 0;
-
-  if (code == MEM) {
-    if (base_n_index_p (XEXP (op, 0)))
-      return 0;
-  }
-
-  return memory_operand (op, mode);
-}
-
-/* Return 1 if OP is a valid R or S operand for an RS, SI or SS type
-   instruction.
+/* Helper routine to implement s_operand and s_imm_operand.
    OP is the current operation.
-   MODE is the current operation mode.  */
+   MODE is the current operation mode.
+   ALLOW_IMMEDIATE specifies whether immediate operands should
+   be accepted or not.  */
 
-int
-r_or_s_operand (op, mode)
+static int
+general_s_operand (op, mode, allow_immediate)
      register rtx op;
      enum machine_mode mode;
+     int allow_immediate;
 {
-  register enum rtx_code code = GET_CODE (op);
+  struct s390_address addr;
 
+  /* Call general_operand first, so that we don't have to
+     check for many special cases.  */
   if (!general_operand (op, mode))
     return 0;
 
-  if (code == MEM) {
-    if (base_n_index_p (XEXP (op, 0)))
-      return 0;
-    else
-      return memory_operand (op, mode);
-  }
-  return register_operand (op, mode);
-}
+  /* Just like memory_operand, allow (subreg (mem ...))
+     after reload.  */
+  if (reload_completed 
+      && GET_CODE (op) == SUBREG 
+      && GET_CODE (SUBREG_REG (op)) == MEM)
+    op = SUBREG_REG (op);
 
-/* Return true if OP is a valid R or S or immediate operand for 
-   RS, SI or SS type instruction.
-   OP is the current operation.
-   MODE is the current operation mode.  */
-
-int
-r_or_s_or_im8_operand (op, mode)
-     register rtx op;
-     enum machine_mode mode;
-{
-  register enum rtx_code code = GET_CODE (op);
+  switch (GET_CODE (op))
+    {
+      /* Constants that we are sure will be forced to the
+         literal pool in reload are OK as s-operand.  Note
+	 that we cannot call s390_preferred_reload_class here
+	 because it might not be known yet at this point 
+	 whether the current function is a leaf or not.  */
+      case CONST_INT:
+      case CONST_DOUBLE:
+	if (!allow_immediate || reload_completed)
+	  break;
+	if (!legitimate_reload_constant_p (op))
+	  return 1;
+	if (!TARGET_64BIT)
+	  return 1;
+	break;
+
+      /* Memory operands are OK unless they already use an
+	 index register.  */
+      case MEM:
+	if (GET_CODE (XEXP (op, 0)) == ADDRESSOF)
+	  return 1;
+	if (s390_decompose_address (XEXP (op, 0), &addr, FALSE) 
+	    && !addr.indx)
+	  return 1;
+	break;
 
-  if (!general_operand (op, mode))
-    return 0;
+      default:
+	break;
+    }
 
-  if (code == MEM) {
-    if (base_n_index_p (XEXP (op, 0)))
-      return 0;
-    else
-      return memory_operand (op, mode);
-  }
-  return register_operand (op, mode) || immediate_operand (op, mode);
+  return 0;
 }
 
-/* Return true if OP is a valid R or X or 16 bit immediate operand for 
-   RX, RR or RI type instruction.
+/* Return true if OP is a valid S-type operand.
    OP is the current operation.
    MODE is the current operation mode.  */
 
 int
-r_or_x_or_im16_operand (op, mode)
+s_operand (op, mode)
      register rtx op;
      enum machine_mode mode;
 {
-
-  if (! general_operand (op, mode))
-    return 0;
-
-  if (GET_CODE (op) == CONST_INT)
-    return (CONST_OK_FOR_LETTER_P (INTVAL (op), 'K'));
-  return register_operand (op, mode) || memory_operand (op, mode);
+  return general_s_operand (op, mode, 0);
 }
 
-/* Return true if OP is a valid R or 8 bit immediate operand.
+/* Return true if OP is a valid S-type operand or an immediate 
+   operand that can be addressed as S-type operand by forcing 
+   it into the literal pool.
    OP is the current operation.
    MODE is the current operation mode.  */
 
 int
-r_or_im8_operand (op, mode)
+s_imm_operand (op, mode)
      register rtx op;
      enum machine_mode mode;
 {
-
-  if (!general_operand (op, mode))
-    return 0;
-
-  if (GET_CODE (op) == CONST_INT)
-    return (CONST_OK_FOR_LETTER_P (INTVAL (op), 'J'));
-  return register_operand (op, mode) || memory_operand (op, mode);
-}
-
-/* Return true if OP is a valid operand for the 'test under mask'
-   instruction with 16 bit immediate.  
-   The value should only have set bits in one halfword.
-   OP is the current operation.
-   MODE is the current operation mode.  */
-
-int
-tmxx_operand (op, mode)
-     register rtx op;
-     enum machine_mode mode ATTRIBUTE_UNUSED;
-{
-  rtx con;
-  if (GET_CODE (op) == CONST_INT)
-    return (CONST_OK_FOR_LETTER_P (INTVAL (op), 'L'));
-  if (GET_CODE (op) == MEM && GET_CODE (XEXP (op, 0)) == SYMBOL_REF &&
-      CONSTANT_POOL_ADDRESS_P (XEXP (op, 0))) 
-    {
-      con = get_pool_constant (XEXP (op, 0));
-
-      if (GET_CODE (con) == CONST_INT)
-	{
-	  unsigned HOST_WIDEST_INT c;
-	  
-	  c = (unsigned HOST_WIDEST_INT) INTVAL (con);
-	  
-	  return ((c & 0xffff) ? ((c & 0xffffffffffff0000ULL)==0) : 
-		  (c & 0xffff0000U) ? ((c & 0xffffffff0000ffffULL)==0) :
-		  (c & 0xffff00000000ULL) ? ((c & 0xffff0000ffffffffULL)==0) :
-		  (c & 0xffff000000000000ULL) ? ((c & 0xffffffffffffULL)==0) : 1);
-		  
-	}
-    }
-  return 0;
+  return general_s_operand (op, mode, 1);
 }
 
 /* Return true if OP is a valid operand for the BRAS instruction.
@@ -682,7 +830,7 @@ load_multiple_operation (op, mode)
   int count = XVECLEN (op, 0);
   unsigned int dest_regno;
   rtx src_addr;
-  int i;
+  int i, off;
 
 
   /* Perform a quick check so we don't blow up below.  */
@@ -695,6 +843,23 @@ load_multiple_operation (op, mode)
   dest_regno = REGNO (SET_DEST (XVECEXP (op, 0, 0)));
   src_addr = XEXP (SET_SRC (XVECEXP (op, 0, 0)), 0);
 
+  /* Check, is base, or base + displacement.  */
+
+  if (GET_CODE (src_addr) == REG)
+    off = 0;
+  else if (GET_CODE (src_addr) == PLUS
+	   && GET_CODE (XEXP (src_addr, 0)) == REG 
+	   && GET_CODE (XEXP (src_addr, 1)) == CONST_INT)
+    {
+      off = INTVAL (XEXP (src_addr, 1));
+      src_addr = XEXP (src_addr, 0);
+    }
+  else
+    return 0;
+
+  if (src_addr == frame_pointer_rtx || src_addr == arg_pointer_rtx)
+    return 0;
+
   for (i = 1; i < count; i++)
     {
       rtx elt = XVECEXP (op, 0, i);
@@ -708,7 +873,8 @@ load_multiple_operation (op, mode)
 	  || GET_CODE (XEXP (SET_SRC (elt), 0)) != PLUS
 	  || ! rtx_equal_p (XEXP (XEXP (SET_SRC (elt), 0), 0), src_addr)
 	  || GET_CODE (XEXP (XEXP (SET_SRC (elt), 0), 1)) != CONST_INT
-	  || INTVAL (XEXP (XEXP (SET_SRC (elt), 0), 1)) != i * 4)
+	  || INTVAL (XEXP (XEXP (SET_SRC (elt), 0), 1))
+	     != off + i * UNITS_PER_WORD)
 	return 0;
     }
 
@@ -725,10 +891,10 @@ store_multiple_operation (op, mode)
      rtx op;
      enum machine_mode mode ATTRIBUTE_UNUSED;
 {
-  int count = XVECLEN (op, 0) - 1;
+  int count = XVECLEN (op, 0);
   unsigned int src_regno;
   rtx dest_addr;
-  int i;
+  int i, off;
 
   /* Perform a quick check so we don't blow up below.  */
   if (count <= 1
@@ -740,6 +906,23 @@ store_multiple_operation (op, mode)
   src_regno = REGNO (SET_SRC (XVECEXP (op, 0, 0)));
   dest_addr = XEXP (SET_DEST (XVECEXP (op, 0, 0)), 0);
 
+  /* Check, is base, or base + displacement.  */
+
+  if (GET_CODE (dest_addr) == REG)
+    off = 0;
+  else if (GET_CODE (dest_addr) == PLUS
+	   && GET_CODE (XEXP (dest_addr, 0)) == REG 
+	   && GET_CODE (XEXP (dest_addr, 1)) == CONST_INT)
+    {
+      off = INTVAL (XEXP (dest_addr, 1));
+      dest_addr = XEXP (dest_addr, 0);
+    }
+  else
+    return 0;
+
+  if (dest_addr == frame_pointer_rtx || dest_addr == arg_pointer_rtx)
+    return 0;
+
   for (i = 1; i < count; i++)
     {
       rtx elt = XVECEXP (op, 0, i);
@@ -753,7 +936,8 @@ store_multiple_operation (op, mode)
 	  || GET_CODE (XEXP (SET_DEST (elt), 0)) != PLUS
 	  || ! rtx_equal_p (XEXP (XEXP (SET_DEST (elt), 0), 0), dest_addr)
 	  || GET_CODE (XEXP (XEXP (SET_DEST (elt), 0), 1)) != CONST_INT
-	  || INTVAL (XEXP (XEXP (SET_DEST (elt), 0), 1)) != i * 4)
+	  || INTVAL (XEXP (XEXP (SET_DEST (elt), 0), 1))
+	     != off + i * UNITS_PER_WORD)
 	return 0;
     }
   return 1;
@@ -800,8 +984,7 @@ int
 legitimate_pic_operand_p (op)
      register rtx op;
 {
-  /* All non-symbolic constants that made it 
-     up to here are fine.  */
+  /* Accept all non-symbolic constants.  */
   if (!SYMBOLIC_CONST (op))
     return 1;
 
@@ -821,13 +1004,7 @@ int
 legitimate_constant_p (op)
      register rtx op;
 {
-  /* Reject doubles and integers out of range.  */
-  if (GET_CODE (op) == CONST_DOUBLE ||
-      (GET_CODE (op) == CONST_INT &&
-       (INTVAL (op) < -32768 || INTVAL (op) > 32767)))
-    return 0;
-
-  /* Accept all other non-symbolic constants.  */
+  /* Accept all non-symbolic constants.  */
   if (!SYMBOLIC_CONST (op))
     return 1;
 
@@ -847,6 +1024,93 @@ legitimate_constant_p (op)
   return 0;
 }
 
+/* Returns true if the constant value OP is a legitimate general
+   operand during and after reload.  The difference to 
+   legitimate_constant_p is that this function will not accept
+   a constant that would need to be forced to the literal pool
+   before it can be used as operand.  */
+
+int
+legitimate_reload_constant_p (op)
+     register rtx op;
+{
+  /* Accept l(g)hi operands.  */
+  if (GET_CODE (op) == CONST_INT
+      && CONST_OK_FOR_LETTER_P (INTVAL (op), 'K'))
+    return 1;
+
+  /* Accept lliXX operands.  */
+  if (TARGET_64BIT
+      && s390_single_hi (op, DImode, 0) >= 0)
+  return 1;
+
+  /* Accept larl operands.  */
+  if (TARGET_64BIT
+      && larl_operand (op, VOIDmode))
+    return 1;
+
+  /* If reload is completed, and we do not already have a
+     literal pool, and OP must be forced to the literal 
+     pool, then something must have gone wrong earlier.
+     We *cannot* force the constant any more, because the
+     prolog generation already decided we don't need to 
+     set up the base register.  */
+  if (reload_completed && !regs_ever_live[BASE_REGISTER])
+    abort ();
+
+  /* Everything else cannot be handled without reload.  */
+  return 0;
+}
+
+/* Given an rtx OP being reloaded into a reg required to be in class CLASS,
+   return the class of reg to actually use.  */
+
+enum reg_class
+s390_preferred_reload_class (op, class)
+     rtx op;
+     enum reg_class class;
+{
+  /* This can happen if a floating point constant is being
+     reloaded into an integer register.  Leave well alone.  */
+  if (GET_MODE_CLASS (GET_MODE (op)) == MODE_FLOAT
+      && class != FP_REGS)
+    return class;
+
+  switch (GET_CODE (op))
+    {
+      /* Constants we cannot reload must be forced into the
+	 literal pool.  For constants we *could* handle directly,
+	 it might still be preferable to put them in the pool and
+	 use a memory-to-memory instruction.
+
+	 However, try to avoid needlessly allocating a literal
+	 pool in a routine that wouldn't otherwise need any.
+	 Heuristically, we assume that 64-bit leaf functions
+	 typically don't need a literal pool, all others do.  */
+      case CONST_DOUBLE:
+      case CONST_INT:
+	if (!legitimate_reload_constant_p (op))
+	  return NO_REGS;
+
+	if (TARGET_64BIT && current_function_is_leaf)
+	  return class;
+
+	return NO_REGS;
+
+      /* If a symbolic constant or a PLUS is reloaded,
+	 it is most likely being used as an address.  */
+      case PLUS:
+      case LABEL_REF:
+      case SYMBOL_REF:
+      case CONST:
+        return ADDR_REGS;
+
+      default:
+	break;
+    }
+
+  return class;
+}
 
 /* Decompose a RTL expression ADDR for a memory address into
    its components, returned in OUT.  The boolean STRICT 
@@ -1063,14 +1327,16 @@ legitimate_la_operand_p (op)
   if (addr.base && GET_CODE (addr.base) == REG)
     {
       if (REGNO (addr.base) == BASE_REGISTER
-          || REGNO (addr.base) == STACK_POINTER_REGNUM)
+	  || REGNO (addr.base) == STACK_POINTER_REGNUM
+	  || REGNO (addr.base) == FRAME_POINTER_REGNUM)
         return TRUE;
     }
 
   if (addr.indx && GET_CODE (addr.indx) == REG)
     {
       if (REGNO (addr.indx) == BASE_REGISTER
-          || REGNO (addr.indx) == STACK_POINTER_REGNUM)
+          || REGNO (addr.indx) == STACK_POINTER_REGNUM
+	  || REGNO (addr.base) == FRAME_POINTER_REGNUM)
         return TRUE;
     }
 
@@ -1364,7 +1630,7 @@ emit_pic_move (operands, mode)
      rtx *operands;
      enum machine_mode mode ATTRIBUTE_UNUSED;
 {
-  rtx temp = reload_in_progress ? operands[0] : gen_reg_rtx (Pmode);
+  rtx temp = no_new_pseudos ? operands[0] : gen_reg_rtx (Pmode);
 
   if (GET_CODE (operands[0]) == MEM && SYMBOLIC_CONST (operands[1]))
     operands[1] = force_reg (Pmode, operands[1]);
@@ -1532,7 +1798,7 @@ print_operand_address (file, addr)
   struct s390_address ad;
 
   if (!s390_decompose_address (addr, &ad, TRUE))
-    output_operand_lossage ("cannot decompose address.\n");
+    output_operand_lossage ("Cannot decompose address.");
  
   if (ad.disp)
     s390_output_symbolic_const (file, ad.disp);
@@ -1657,15 +1923,26 @@ print_operand (file, x, code)
 
     case CONST_INT:
       if (code == 'b')
-        fprintf (file, "%d", (int)(INTVAL (x) & 0xff));
-      else if (code == 'X')
-        fprintf (file, "%d", (int)(INTVAL (x) & 0xff));
+        fprintf (file, HOST_WIDE_INT_PRINT_DEC, INTVAL (x) & 0xff);
+      else if (code == 'x')
+        fprintf (file, HOST_WIDE_INT_PRINT_DEC, INTVAL (x) & 0xffff);
+      else if (code == 'h')
+        fprintf (file, HOST_WIDE_INT_PRINT_DEC, ((INTVAL (x) & 0xffff) ^ 0x8000) - 0x8000);
+      else
+        fprintf (file, HOST_WIDE_INT_PRINT_DEC, INTVAL (x));
+      break;
+
+    case CONST_DOUBLE:
+      if (GET_MODE (x) != VOIDmode)
+        abort ();
+      if (code == 'b')
+        fprintf (file, HOST_WIDE_INT_PRINT_DEC, CONST_DOUBLE_LOW (x) & 0xff);
       else if (code == 'x')
-        fprintf (file, "0x%x", (int)(INTVAL (x) & 0xffff));
+        fprintf (file, HOST_WIDE_INT_PRINT_DEC, CONST_DOUBLE_LOW (x) & 0xffff);
       else if (code == 'h')
-        fprintf (file, "%d", (int)(INTVAL (x) << 16) >> 16);
+        fprintf (file, HOST_WIDE_INT_PRINT_DEC, ((CONST_DOUBLE_LOW (x) & 0xffff) ^ 0x8000) - 0x8000);
       else
-        fprintf (file, "%d", (int)INTVAL (x));
+        abort ();
       break;
 
     default:
@@ -1694,6 +1971,13 @@ reg_used_in_mem_p (regno, x)
 			     XEXP (x, 0), 0))
 	return 1;
     }
+  else if (code == SET 
+	   && GET_CODE (SET_DEST (x)) == PC)
+    {
+      if (refers_to_regno_p (regno, regno+1,
+			     SET_SRC (x), 0))
+	return 1;
+    }
 
   fmt = GET_RTX_FORMAT (code);
   for (i = GET_RTX_LENGTH (code) - 1; i >= 0; i--)
@@ -1718,7 +2002,7 @@ addr_generation_dependency_p (dep_rtx, insn)
      rtx dep_rtx; 
      rtx insn;
 {
-  rtx target;
+  rtx target, pat;
 
   if (GET_CODE (dep_rtx) == SET)
     {
@@ -1729,8 +2013,19 @@ addr_generation_dependency_p (dep_rtx, insn)
 	  int regno = REGNO (target);
 
 	  if (get_attr_type (insn) == TYPE_LA)
-	    return refers_to_regno_p (regno, regno+1,
-				      SET_SRC (PATTERN (insn)), 0);
+	    {
+	      pat = PATTERN (insn);
+	      if (GET_CODE (pat) == PARALLEL)
+		{
+		  if (XVECLEN (pat, 0) != 2)
+		    abort();
+		  pat = XVECEXP (pat, 0, 0);
+		}
+	      if (GET_CODE (pat) == SET)
+		return refers_to_regno_p (regno, regno+1, SET_SRC (pat), 0);
+	      else
+		abort();
+	    }
 	  else if (get_attr_atype (insn) == ATYPE_MEM)
 	    return reg_used_in_mem_p (regno, PATTERN (insn));
 	}
@@ -1839,6 +2134,14 @@ s390_adjust_priority (insn, priority)
       if (priority >= 0 && priority < 0x01000000)
 	priority <<= 3;
       break;
+    case TYPE_LM:
+      /* LM in epilogue should never be scheduled. This
+	 is due to literal access done in function body.
+	 The usage of register 13 is not mentioned explicitly,
+	 leading to scheduling 'LM' accross this instructions.  
+      */ 
+      priority = 0x7fffffff;
+      break;
     }
   
   return priority;
@@ -2114,8 +2417,6 @@ s390_final_chunkify (chunkify)
   int addr, naddr = 0, uids;
   int chunk_max = 0;
 
-  const char *asms;
-
   int size = insn_current_address;
 
   int *ltorg_uids;
@@ -2171,19 +2472,6 @@ s390_final_chunkify (chunkify)
 	      warning ("no code label found");
 	    }
 	} 
-      else if (GET_CODE (PATTERN (insn)) == ASM_INPUT && !TARGET_64BIT) 
-	{
-	  asms = XSTR (PATTERN (insn),0);
-	  
-	  if ((memcmp (asms,".section",8) == 0) ||
-	      (memcmp (asms,".text",5) == 0)    ||
-	      (memcmp (asms,"\t.section",9) == 0) ||
-	      (memcmp (asms,"\t.text",6) == 0))  {
-	    ltorg_uids[max_ltorg++] = INSN_UID (insn);
-	    INSN_ADDRESSES_NEW (emit_insn_before (gen_rtx_ASM_INPUT (VOIDmode,
-					   ".align 4"), insn), -1);
-	  }
-	}
     }
   ltorg_uids[max_ltorg] = 0;
   for (insn=get_insns (),uids=0; insn;insn = next_real_insn (insn)) 
@@ -2253,19 +2541,6 @@ s390_dump_literal_pool (act_insn, stop)
   function_section (current_function_decl);
 }
 
-
-#ifdef DWARF2_DEBUGGING_INFO
-extern char *dwarf2out_cfi_label PARAMS ((void));
-#endif
-
-/* Flag set in prologue, used in epilog to know
-   if stack is allocated or not.  */
-static int leaf_function_flag;
-
-/* Symbol references needed by the profile code;
-   set up by the function prologue routine if necessary.  */
-rtx s390_profile[10];
-
 /* Number of elements of current constant pool.  */
 int s390_nr_constants;
 
@@ -2285,111 +2560,14 @@ save_fprs_p ()
   return 0;
 }
 
-/* Return true if urrent function is a leaf function, 
-   without automatics, alloca or vararg stuff.  */
-
-static int
-cur_is_leaf_function ()
-{
-  int lsize =  get_frame_size () + current_function_outgoing_args_size
-    + save_fprs_p () * 64;
-
-  if (leaf_function_p () && ((lsize) == 0) &&
-      ! (current_function_calls_alloca) &&
-      ! (current_function_stdarg) && ! (current_function_varargs))
-    return 1;
-  return 0;
-}
-
-/* Return offset between argument pointer and frame pointer 
-   initially after prologue.  */
-
-int 
-s390_arg_frame_offset ()
-{
-  int lsize =  get_frame_size () + current_function_outgoing_args_size
-    + save_fprs_p () * 64;
-
-  if (cur_is_leaf_function ())
-    return STACK_POINTER_OFFSET;
-  else
-    return 2*STACK_POINTER_OFFSET + lsize;
-}
-
-/* Output code to stdio stream FILE to save floating point 
-   registers on current stack, at offset OFFSET to the frame
-   pointer register FP.  */
-
-static int 
-save_fprs (file, offset, fp)
-     FILE *file;
-     long offset;
-     int fp;
-{
-  int i;
-
-  if (!TARGET_64BIT)
-    return 0;
-
-  for (i=24; i<=31; i++) 
-    {
-      if (regs_ever_live[i] == 1)
-	{
-	  fprintf (file, "\tstd\t%s,%ld(%s)\n", reg_names[i], 
-		   (i-24) * 8 + offset, reg_names[fp]); 
-	}
-    }
-
-  return 1;
-}
-
-/* Output code to stdio stream FILE to restore floating point 
-   registers from current stack, at offset OFFSET to the frame
-   pointer register FP.  */
-
-static int 
-restore_fprs (file, offset, fp)
-     FILE *file;
-     long offset;
-     int fp;
-{
-  int i;
-
-  if (!TARGET_64BIT)
-    return 0;
-
-  if (!save_fprs_p ())
-    return 0;
-
-  if (offset < 0) 
-    {
-      fp = 1;
-      offset = 0;
-      fprintf (file, "\tlgr\t%s,%s\n", reg_names[fp], 
-	       reg_names[STACK_POINTER_REGNUM]); 
-      fprintf (file, "\taghi\t%s,-64\n", reg_names[fp]); 
-    }
-
-  for (i=24; i<=31; i++) 
-    {
-      if (regs_ever_live[i] == 1)
-	{
-	  fprintf (file, "\tld\t%s,%ld(%s)\n", reg_names[i], 
-		   (i-24) * 8 + offset, reg_names[fp]); 
-	}
-    }
-
-  return 1;
-}
-
 /* Output main constant pool to stdio stream FILE.  */ 
 
-static void
+void
 s390_output_constant_pool (file)
      FILE *file;
 {
   /* Output constant pool.  */
-  if (s390_nr_constants || regs_ever_live[BASE_REGISTER])
+  if (s390_nr_constants)
     {
       s390_pool_count = 0;
       if (TARGET_64BIT)
@@ -2410,66 +2588,150 @@ s390_output_constant_pool (file)
 	       s390_pool_count);
       if (TARGET_64BIT)
 	function_section (current_function_decl);
-      
-      regs_ever_live[BASE_REGISTER] = 1;
     }
 }
 
-/* Add constant CTX to the constant pool at a late time 
-   (after the initial pass to count the number of constants
-   was already done).  Returns the resulting constant 
-   pool reference.  */
+/* Find first call clobbered register unsused in a function.
+   This could be used as base register in a leaf function
+   or for holding the return address before epilogue.  */
 
-static rtx
-s390_force_const_mem_late (cst)
-     rtx cst;
+static int
+find_unused_clobbered_reg ()
+{
+  int i;
+  for (i = 0; i < 6; i++)
+    if (!regs_ever_live[i])
+      return i;
+  return 0;
+}
+
+/* Fill FRAME with info about frame of current function.  */
+
+static void
+s390_frame_info (frame)
+     struct s390_frame *frame;
+{
+  int i, j;
+  HOST_WIDE_INT fsize = get_frame_size ();
+
+  if (fsize > 0x7fff0000)
+    fatal_error ("Total size of local variables exceeds architecture limit.");
+
+  /* fprs 8 - 15 are caller saved for 64 Bit ABI.  */
+  frame->save_fprs_p = save_fprs_p ();
+
+  frame->frame_size = fsize + frame->save_fprs_p * 64;
+
+  /* Does function need to setup frame and save area.  */
+  
+  if (! current_function_is_leaf
+      || frame->frame_size > 0
+      || current_function_calls_alloca 
+      || current_function_stdarg
+      || current_function_varargs)
+    frame->frame_size += STARTING_FRAME_OFFSET;
+
+  /* If we need to allocate a frame, the stack pointer is changed.  */ 
+
+  if (frame->frame_size > 0)
+    regs_ever_live[STACK_POINTER_REGNUM] = 1;
+
+  /* If there is (possibly) any pool entry, we need to 
+     load base register.  */
+
+  if (get_pool_size () 
+      || !CONST_OK_FOR_LETTER_P (frame->frame_size, 'K')
+      || (!TARGET_64BIT && current_function_uses_pic_offset_table))
+    regs_ever_live[BASE_REGISTER] = 1; 
+
+  /* If we need the GOT pointer, remember to save/restore it.  */
+
+  if (current_function_uses_pic_offset_table)
+    regs_ever_live[PIC_OFFSET_TABLE_REGNUM] = 1;
+
+  /* Frame pointer needed.   */
+    
+  frame->frame_pointer_p = frame_pointer_needed;
+
+  /* Find first and last gpr to be saved.  */
+  
+  for (i = 6; i < 16; i++)
+    if (regs_ever_live[i])
+      break;
+
+  for (j = 15; j > i; j--)
+    if (regs_ever_live[j])
+      break;
+  
+  if (i == 16)
+    {
+      /* Nothing to save / restore.  */ 
+      frame->first_save_gpr = -1;
+      frame->first_restore_gpr = -1;
+      frame->last_save_gpr = -1;
+      frame->return_reg_saved_p = 0;
+    }
+  else
+    {
+      /* Save / Restore from gpr i to j.  */
+      frame->first_save_gpr = i;
+      frame->first_restore_gpr = i;
+      frame->last_save_gpr  = j;
+      frame->return_reg_saved_p = (j >= RETURN_REGNUM && i <= RETURN_REGNUM);
+    }
+
+  if (current_function_stdarg || current_function_varargs)
+    {
+      /* Varargs function need to save from gpr 2 to gpr 15.  */
+      frame->first_save_gpr = 2;
+    }
+}
+
+/* Return offset between argument pointer and frame pointer 
+   initially after prologue.  */
+
+int 
+s390_arg_frame_offset ()
 {
-  cst = force_const_mem (Pmode, cst);
+  struct s390_frame frame;
 
-  s390_nr_constants++;
-  regs_ever_live[BASE_REGISTER] = 1;
+  /* Compute frame_info.  */
 
-  emit_insn_before (gen_rtx (USE, Pmode, cst), get_insns ());
+  s390_frame_info (&frame);
 
-  return cst;
+  return frame.frame_size + STACK_POINTER_OFFSET;
 }
 
-/* Add a reference to the symbol NAME to the constant pool.
-   FUNC specifies whether NAME refers to a function, while
-   GLOBAL specifies whether NAME is a global symbol.  Depending
-   on these flags, the appopriate PLT or GOT references are
-   generated.  Returns the constant pool reference.  */
+/* Emit insn to save fpr REGNUM at offset OFFSET relative
+   to register BASE.  Return generated insn.  */ 
 
 static rtx
-s390_force_const_mem_symbol (name, func, global)
-     const char *name;
-     int func;
-     int global;
+save_fpr (base, offset, regnum)
+     rtx base;
+     int offset;
+     int regnum;     
 {
-  rtx symbol;
+  rtx addr;
+  addr = gen_rtx_MEM (DFmode, plus_constant (base, offset));
+  set_mem_alias_set (addr, s390_sr_alias_set);
 
-  if (TARGET_64BIT)
-    abort ();
+  return emit_move_insn (addr, gen_rtx_REG (DFmode, regnum));
+}
 
-  symbol = gen_rtx (SYMBOL_REF, Pmode, name);
-  SYMBOL_REF_FLAG (symbol) = !global;
+/* Emit insn to restore fpr REGNUM from offset OFFSET relative
+   to register BASE.  Return generated insn.  */ 
 
-  if (flag_pic)
-    {
-      if (global)
-        {
-          current_function_uses_pic_offset_table = 1;
-          symbol = gen_rtx_UNSPEC (VOIDmode, gen_rtvec (1, symbol), func? 114 : 112);
-          symbol = gen_rtx_CONST (VOIDmode, symbol);
-        }
-      else
-        {
-          symbol = gen_rtx_UNSPEC (VOIDmode, gen_rtvec (1, symbol), 100);
-          symbol = gen_rtx_CONST (VOIDmode, symbol);
-        }
-    }
+static rtx
+restore_fpr (base, offset, regnum)
+     rtx base;
+     int offset;
+     int regnum;
+{
+  rtx addr;
+  addr = gen_rtx_MEM (DFmode, plus_constant (base, offset));
+  set_mem_alias_set (addr, s390_sr_alias_set);
 
-  return s390_force_const_mem_late (symbol);
+  return emit_move_insn (gen_rtx_REG (DFmode, regnum), addr);
 }
 
 /* Output the function prologue assembly code to the 
@@ -2478,401 +2740,396 @@ s390_force_const_mem_symbol (name, func, global)
 
 void
 s390_function_prologue (file, lsize)
-     FILE *file;
-     HOST_WIDE_INT lsize;
+     FILE *file ATTRIBUTE_UNUSED;
+     HOST_WIDE_INT lsize ATTRIBUTE_UNUSED;
 {
-  extern int profile_label_no;
-  int i, j;
-  long frame_size;
-  rtx stack_label = 0, got_label = 0;
-  char *l;
-  const char *const b64 = TARGET_64BIT ? "g" : "";
+  if (get_pool_size () > S390_POOL_MAX)
+    s390_final_chunkify (1);
+  else
+    s390_final_chunkify (0);
+}
 
-  /* Check for too large size of local variables */
+/* Output the function epilogue assembly code to the 
+   stdio stream FILE.  The local frame size is passed
+   in LSIZE.  */
 
-  if (lsize > 0x7fff0000)
-    fatal_error ("total size of local variables exceeds architecture limit");
+void
+s390_function_epilogue (file, lsize)
+     FILE *file ATTRIBUTE_UNUSED;
+     HOST_WIDE_INT lsize ATTRIBUTE_UNUSED;
+{
+  current_function_uses_pic_offset_table = 0;
+  s390_pool_start_insn = NULL_RTX;
+  s390_pool_count = -1;
+  s390_function_count++;
+}
 
-  /* Profile code (-p, -a, -ax needs some literals).  */
+/* Expand the prologue into a bunch of separate insns.  */
 
-  if (profile_flag && !TARGET_64BIT)
-    {
-      static char label[128];
-      sprintf (label, "%sP%d", LPREFIX, profile_label_no);
+void
+s390_emit_prologue ()
+{
+  struct s390_frame frame;
+  rtx insn, addr;
+  rtx temp_reg;
+  int i, limit;
 
-      s390_profile[4] = s390_force_const_mem_symbol ("_mcount", 1, 1);
-      s390_profile[9] = s390_force_const_mem_symbol (label, 0, 0);
-    }
+  /* Compute frame_info.  */
 
-  if (get_pool_size () > S390_POOL_MAX)
-    s390_final_chunkify (1);
+  s390_frame_info (&frame);
+
+  /* Choose best register to use for temp use within prologue.  */
+  
+  if (frame.return_reg_saved_p
+      && !has_hard_reg_initial_val (Pmode, RETURN_REGNUM))
+    temp_reg = gen_rtx_REG (Pmode, RETURN_REGNUM);
   else
-    s390_final_chunkify (0);
+    temp_reg = gen_rtx_REG (Pmode, 1);
 
-  if (current_function_uses_pic_offset_table)
-    regs_ever_live[12] = 1;
+  /* Save call saved gprs.  */
 
-  if (!TARGET_64BIT && current_function_uses_pic_offset_table)
+  if (frame.first_save_gpr != -1)
     {
-      got_label = s390_force_const_mem_symbol ("_GLOBAL_OFFSET_TABLE_", 0, 0);
-    }
+      addr = plus_constant (stack_pointer_rtx, 
+			    frame.first_save_gpr * UNITS_PER_WORD);
+      addr = gen_rtx_MEM (Pmode, addr);
+      set_mem_alias_set (addr, s390_sr_alias_set);
 
-  if ((frame_size = 
-       STARTING_FRAME_OFFSET + lsize + save_fprs_p () * 64) > 0x7fff)
-    {
-      stack_label = s390_force_const_mem_late (GEN_INT (frame_size));
-    }
+      if (frame.first_save_gpr != frame.last_save_gpr )
+	{
+	  insn = emit_insn (gen_store_multiple (addr,
+			      gen_rtx_REG (Pmode, frame.first_save_gpr),
+			      GEN_INT (frame.last_save_gpr 
+				       - frame.first_save_gpr + 1)));
 
-  if (!optimize)
-    {
-      /* Stupid register allocation is stupid ...
-         It does not always recognize the base register is used.  */
-      
-      regs_ever_live[BASE_REGISTER] = 1;
-    }
-
- if (cur_is_leaf_function ())
-   {
-      leaf_function_flag = 1;
-      fprintf (file, "%s\tleaf function\n", ASM_COMMENT_START);
-      fprintf (file, "%s\thas varargs             %d\n", ASM_COMMENT_START,
-	       current_function_stdarg);
-      fprintf (file, "%s\tincoming args (stack)   %d\n", ASM_COMMENT_START,
-	       current_function_args_size);
-      fprintf (file, "%s\tfunction length         %d\n", ASM_COMMENT_START,
-	       insn_current_address);
-      fprintf (file, "%s\tregister live           ", ASM_COMMENT_START);
-      for (i = 0; i < FIRST_PSEUDO_REGISTER; i++)
-	fprintf (file, "%d", regs_ever_live[i]);
-      fputc   ('\n',file);
-      
-      /* Save gprs 6 - 15 and fprs 4 and 6.  */
-      for (i = 6; i < 13 && (regs_ever_live[i] == 0); i++);
+	  /* Set RTX_FRAME_RELATED_P for all sets within store multiple.  */
 
-      if (s390_nr_constants || regs_ever_live[13] || i != 13)
-	{
-	  fprintf (file, "\tstm%s\t%s,%s,%d(%s)\n", 
-                         b64, reg_names[i], reg_names[13],
-                         i * UNITS_PER_WORD,
-		         reg_names[STACK_POINTER_REGNUM]);
-#ifdef INCOMING_RETURN_ADDR_RTX
-	  if (dwarf2out_do_frame ())
+	  limit = XVECLEN (PATTERN (insn), 0);
+	  
+	  for (i = 0; i < limit; i++)
 	    {
-	      l = dwarf2out_cfi_label ();
-	      dwarf2out_def_cfa (l, STACK_POINTER_REGNUM, 
-				 STACK_POINTER_OFFSET);
-	      for (j = i; j <= 14; j++)
-		dwarf2out_reg_save (l, j, (TARGET_64BIT ? (j-20) : (j-24))
-				    * UNITS_PER_WORD);
-	      if (regs_ever_live[18])
-		dwarf2out_reg_save (l, 18, -16);
-	      if (regs_ever_live[19])
-		dwarf2out_reg_save (l, 19, -8);
+	      rtx x = XVECEXP (PATTERN (insn), 0, i);
+	      
+	      if (GET_CODE (x) == SET)
+		RTX_FRAME_RELATED_P (x) = 1;
 	    }
-#endif
 	}
-
-      s390_output_constant_pool (file);
-
-      /* Save fprs.  */
-
-      if (!TARGET_64BIT)
+      else
 	{
-	  if (regs_ever_live[18])
-	    fprintf (file, "\tstd\t4,80(%s)\n", reg_names[STACK_POINTER_REGNUM]);
-	  if (regs_ever_live[19])
-	    fprintf (file, "\tstd\t6,88(%s)\n", reg_names[STACK_POINTER_REGNUM]);
+	  insn = emit_move_insn (addr, 
+				 gen_rtx_REG (Pmode, frame.first_save_gpr));
 	}
+      RTX_FRAME_RELATED_P (insn) = 1;
     }
-  else
-    {				/* No leaf function.  */
-      fprintf (file, "%s\tleaf function           %d\n", ASM_COMMENT_START,
-	       leaf_function_p ());
-      fprintf (file, "%s\tautomatics              %d\n", ASM_COMMENT_START,
-	       (int)lsize);
-      fprintf (file, "%s\toutgoing args           %d\n", ASM_COMMENT_START,
-	       current_function_outgoing_args_size);
-      fprintf (file, "%s\tneed frame pointer      %d\n", ASM_COMMENT_START,
-	       frame_pointer_needed);
-      fprintf (file, "%s\tcall alloca             %d\n", ASM_COMMENT_START,
-	       current_function_calls_alloca);
-      fprintf (file, "%s\thas varargs             %d\n", ASM_COMMENT_START,
-	       current_function_stdarg || current_function_varargs);
-      fprintf (file, "%s\tincoming args (stack)   %d\n", ASM_COMMENT_START,
-	       current_function_args_size);
-      fprintf (file, "%s\tfunction length         %d\n", ASM_COMMENT_START,
-	       insn_current_address);
-      fprintf (file, "%s\tregister live           ", ASM_COMMENT_START);
-      for (i = 0; i < FIRST_PSEUDO_REGISTER; i++)
-	fprintf (file, "%d", regs_ever_live[i]);
-      fputc   ('\n',file);
-
-      /* Save gprs 6 - 15 and fprs 4 and 6.  */
+
+  /* Dump constant pool and set constant pool register (13).  */
+ 
+  insn = emit_insn (gen_lit ());
+  
+  /* Save fprs for variable args.  */
+
+  if (current_function_stdarg || current_function_varargs)
+    {
+      /* Save fpr 0 and 2.  */ 
+
+      save_fpr (stack_pointer_rtx, STACK_POINTER_OFFSET - 32, 16); 
+      save_fpr (stack_pointer_rtx, STACK_POINTER_OFFSET - 24, 17); 
       
-      if (current_function_stdarg || current_function_varargs)
-	{
-	  i = 2;
-	}
-      else
+      if (TARGET_64BIT)
 	{
-	  for (i = 6; i < 13 && (regs_ever_live[i] == 0); i++);
+	  /* Save fpr 4 and 6.  */
+ 
+	  save_fpr (stack_pointer_rtx, STACK_POINTER_OFFSET - 16, 18); 
+	  save_fpr (stack_pointer_rtx, STACK_POINTER_OFFSET - 8, 19); 
 	}
+    }
 
-      fprintf (file, "\tstm%s\t%s,%s,%d(%s)\n", 
-                     b64, reg_names[i], reg_names[15], i * UNITS_PER_WORD,
-	             reg_names[STACK_POINTER_REGNUM]);
+  /* Save fprs 4 and 6 if used (31 bit ABI).  */
 
-#ifdef INCOMING_RETURN_ADDR_RTX
-      if (dwarf2out_do_frame ())
+  if (!TARGET_64BIT)
+    {
+      /* Save fpr 4 and 6.  */
+      if (regs_ever_live[18])
 	{
-	  l = dwarf2out_cfi_label ();
-	  dwarf2out_def_cfa (l, STACK_POINTER_REGNUM, STACK_POINTER_OFFSET);
-	  for (j = i; j <= 15; j++)
-	    dwarf2out_reg_save (l, j, (TARGET_64BIT ? (j-20) : (j-24)) *
-				UNITS_PER_WORD);
-	  if (regs_ever_live[18])
-	    dwarf2out_reg_save (l, 18, -16);
-	  if (regs_ever_live[19])
-	    dwarf2out_reg_save (l, 19, -8);
+	  insn = save_fpr (stack_pointer_rtx, STACK_POINTER_OFFSET - 16, 18);
+	  RTX_FRAME_RELATED_P (insn) = 1;
 	}
-#endif
+      if (regs_ever_live[19]) 
+	{
+	  insn = save_fpr (stack_pointer_rtx, STACK_POINTER_OFFSET - 8, 19); 
+	  RTX_FRAME_RELATED_P (insn) = 1;
+	}
+    }
 
-      s390_output_constant_pool (file);
+  /* Decrement stack pointer.  */
 
-      /* Save fprs.  */
+  if (frame.frame_size > 0)
+    {
+      rtx frame_off = GEN_INT (-frame.frame_size);
 
-      if (current_function_stdarg || current_function_varargs)
-	{
-	  fprintf (file, "\tstd\t%s,%d(%s)\n", 
-		   reg_names[16],
-		   STACK_POINTER_OFFSET-32,
-		   reg_names[STACK_POINTER_REGNUM]);
-	  fprintf (file, "\tstd\t%s,%d(%s)\n",
-		   reg_names[17],
-		   STACK_POINTER_OFFSET-24,
-		   reg_names[STACK_POINTER_REGNUM]);
-	  if (TARGET_64BIT)
-	    {
-	      fprintf (file, "\tstd\t%s,%d(%s)\n", 
-		       reg_names[18],
-		       STACK_POINTER_OFFSET-16,
-		       reg_names[STACK_POINTER_REGNUM]);
-	      fprintf (file, "\tstd\t%s,%d(%s)\n", 
-		       reg_names[19],
-		       STACK_POINTER_OFFSET-8,
-		       reg_names[STACK_POINTER_REGNUM]);
-	    }
-	}
-      if (!TARGET_64BIT)
+      /* Save incoming stack pointer into temp reg.  */
+      
+      if (TARGET_BACKCHAIN || frame.save_fprs_p)
 	{
-	  if (regs_ever_live[18])
-	    fprintf (file, "\tstd\t%s,%d(%s)\n", 
-		     reg_names[18],
-		     STACK_POINTER_OFFSET-16,
-		     reg_names[STACK_POINTER_REGNUM]);
-	  if (regs_ever_live[19])
-	    fprintf (file, "\tstd\t%s,%d(%s)\n",
-		     reg_names[19],
-		     STACK_POINTER_OFFSET-8,
-		     reg_names[STACK_POINTER_REGNUM]);
+	  insn = emit_insn (gen_move_insn (temp_reg, stack_pointer_rtx));
 	}
-
       
-      if (save_fprs_p () && frame_size > 4095) 
+      /* Substract frame size from stack pointer.  */
+
+      frame_off = GEN_INT (-frame.frame_size);
+      if (!CONST_OK_FOR_LETTER_P (-frame.frame_size, 'K'))
+	frame_off = force_const_mem (Pmode, frame_off);
+
+      insn = emit_insn (gen_add2_insn (stack_pointer_rtx, frame_off));
+      RTX_FRAME_RELATED_P (insn) = 1;
+      REG_NOTES (insn) = 
+	gen_rtx_EXPR_LIST (REG_FRAME_RELATED_EXPR,
+			   gen_rtx_SET (VOIDmode, stack_pointer_rtx,
+				   gen_rtx_PLUS (Pmode, stack_pointer_rtx,
+			           GEN_INT (-frame.frame_size))),
+			   REG_NOTES (insn));
+
+      /* Set backchain.  */
+      
+      if (TARGET_BACKCHAIN)
 	{
-	  int fp = 1;
-	  fprintf (file, "\tlgr\t%s,%s\n", reg_names[fp], 
-		   reg_names[STACK_POINTER_REGNUM]); 
-	  fprintf (file, "\taghi\t%s,-64\n", reg_names[fp]);
-	  save_fprs (file, 0, fp);
+	  addr = gen_rtx_MEM (Pmode, stack_pointer_rtx);
+	  set_mem_alias_set (addr, s390_sr_alias_set);
+	  insn = emit_insn (gen_move_insn (addr, temp_reg));
 	}
+    }
 
-      /* Decrement stack.  */
+  /* Save fprs 8 - 15 (64 bit ABI).  */
+  
+  if (frame.save_fprs_p)
+    {
+      insn = emit_insn (gen_add2_insn (temp_reg, GEN_INT(-64)));
 
-      if (TARGET_BACKCHAIN || (frame_size + STACK_POINTER_OFFSET > 4095
-			       || frame_pointer_needed
-			       || current_function_calls_alloca))
-	{
+      for (i = 24; i < 32; i++)
+	if (regs_ever_live[i])
+	  {
+	    rtx addr = plus_constant (stack_pointer_rtx, 
+				      frame.frame_size - 64 + (i-24)*8);
+
+	    insn = save_fpr (temp_reg, (i-24)*8, i);
+	    RTX_FRAME_RELATED_P (insn) = 1;
+	    REG_NOTES (insn) = 
+	      gen_rtx_EXPR_LIST (REG_FRAME_RELATED_EXPR,
+		gen_rtx_SET (VOIDmode, 
+			     gen_rtx_MEM (DFmode, addr),
+			     gen_rtx_REG (DFmode, i)),
+		REG_NOTES (insn));
+	  }
+    }
+	    
+  /* Set frame pointer, if needed.  */
+  
+  if (frame.frame_pointer_p)
+    {
+      insn = emit_move_insn (hard_frame_pointer_rtx, stack_pointer_rtx);
+      RTX_FRAME_RELATED_P (insn) = 1;
+    }
 
-	  fprintf (file, "\tl%sr\t%s,%s\n", b64, 
-			 reg_names[1], reg_names[STACK_POINTER_REGNUM]);
-	}
+  /* Set up got pointer, if needed.  */
+  
+  if (current_function_uses_pic_offset_table)
+    {
+      rtx got_symbol = gen_rtx_SYMBOL_REF (Pmode, "_GLOBAL_OFFSET_TABLE_");
+      SYMBOL_REF_FLAG (got_symbol) = 1;
 
-      if (stack_label)
+      if (TARGET_64BIT)
 	{
-	  rtx operands[2];
+	  insn = emit_insn (gen_movdi (pic_offset_table_rtx,
+				       got_symbol));		 
 
-	  operands[0] = stack_pointer_rtx;
-	  operands[1] = stack_label;
-	  if (TARGET_64BIT)
-	    output_asm_insn ("sg\t%0,%1", operands);
-	  else
-	    output_asm_insn ("s\t%0,%1", operands);
+          /* It can happen that the GOT pointer isn't really needed ...  */
+          REG_NOTES(insn) = gen_rtx_EXPR_LIST (REG_MAYBE_DEAD, NULL_RTX,
+                                               REG_NOTES (insn));
 	}
       else
 	{
-	  fprintf (file, "\ta%shi\t%s,-%ld\n",b64, 
-		   reg_names[STACK_POINTER_REGNUM], frame_size);
+          got_symbol = gen_rtx_UNSPEC (VOIDmode, 
+				       gen_rtvec (1, got_symbol), 100);
+          got_symbol = gen_rtx_CONST (VOIDmode, got_symbol);
+	  got_symbol = force_const_mem (Pmode, got_symbol);
+	  insn = emit_move_insn (pic_offset_table_rtx,
+				 got_symbol);		 
+	  insn = emit_insn (gen_add2_insn (pic_offset_table_rtx,
+					   gen_rtx_REG (Pmode, BASE_REGISTER)));
+
+	  /* We need the GOT pointer even if we don't know it ...  */
+	  emit_insn (gen_rtx_USE (VOIDmode, pic_offset_table_rtx));
 	}
-#ifdef INCOMING_RETURN_ADDR_RTX
-      if (dwarf2out_do_frame ())
-	{
-	  if (frame_pointer_needed)
-	    dwarf2out_def_cfa ("", HARD_FRAME_POINTER_REGNUM,
-			       STACK_POINTER_OFFSET+frame_size);
-	  else
-	    dwarf2out_def_cfa ("", STACK_POINTER_REGNUM,
-			       STACK_POINTER_OFFSET+frame_size);
-	}
-#endif
+    }      
+}
 
+/* Expand the epilogue into a bunch of separate insns. */
 
-      /* Generate backchain.  */
+void
+s390_emit_epilogue ()
+{
+  struct s390_frame frame;
+  rtx frame_pointer, return_reg;
+  int area_bottom, area_top, offset;
+  rtvec p;
 
-      if (TARGET_BACKCHAIN || (frame_size + STACK_POINTER_OFFSET > 4095
-			       || frame_pointer_needed
-			       || current_function_calls_alloca))
-	{
-	  fprintf (file, "\tst%s\t%s,0(%s)\n", 
-                         b64, reg_names[1], reg_names[STACK_POINTER_REGNUM]);
-	}
-    }
+  /* Compute frame_info.  */
+ 
+  s390_frame_info (&frame);
+
+  /* Check whether to use frame or stack pointer for restore.  */
+
+  frame_pointer = frame.frame_pointer_p ? 
+    hard_frame_pointer_rtx : stack_pointer_rtx;
+
+  /* Compute which parts of the save area we need to access.  */
 
-  if (frame_pointer_needed)
+  if (frame.first_restore_gpr != -1)
     {
-      fprintf (file, "\tl%sr\t%s,%s\n", b64, 
-                     reg_names[FRAME_POINTER_REGNUM], 
-                     reg_names[STACK_POINTER_REGNUM]);
+      area_bottom = frame.first_restore_gpr * UNITS_PER_WORD;
+      area_top = (frame.last_save_gpr + 1) * UNITS_PER_WORD;
+    }
+  else
+    {
+      area_bottom = INT_MAX;
+      area_top = INT_MIN;
     }
 
-  /* Load GOT if used and emit use insn that optimizer does not
-     erase literal pool entry.  */
-
-  if (current_function_uses_pic_offset_table)
+  if (TARGET_64BIT)
     {
-      rtx operands[3];
-      if (TARGET_64BIT)
+      if (frame.save_fprs_p)
 	{
-	  fprintf (file, "\tlarl\t%s,_GLOBAL_OFFSET_TABLE_\n",
-			 reg_names[PIC_OFFSET_TABLE_REGNUM]);
+	  if (area_bottom > -64)
+	    area_bottom = -64;
+	  if (area_top < 0)
+	    area_top = 0;
 	}
-      else
+    }
+  else
+    {
+      if (regs_ever_live[18])
 	{
-	  operands[0] = gen_rtx (REG, Pmode, PIC_OFFSET_TABLE_REGNUM);
-	  operands[1] = got_label;
-	  operands[2] = gen_rtx (REG, Pmode, BASE_REGISTER);
-	  output_asm_insn ("l\t%0,%1\n\tar\t%0,%2", operands);
+	  if (area_bottom > STACK_POINTER_OFFSET - 16)
+	    area_bottom = STACK_POINTER_OFFSET - 16;
+	  if (area_top < STACK_POINTER_OFFSET - 8)
+	    area_top = STACK_POINTER_OFFSET - 8;
+	}
+      if (regs_ever_live[19])
+	{
+	  if (area_bottom > STACK_POINTER_OFFSET - 8)
+	    area_bottom = STACK_POINTER_OFFSET - 8;
+	  if (area_top < STACK_POINTER_OFFSET)
+	    area_top = STACK_POINTER_OFFSET;
 	}
     }
-  /* Save FPRs below save area.  */
 
-  if (frame_size <= 4095)
-    save_fprs (file, frame_size - 64, STACK_POINTER_REGNUM);
+  /* Check whether we can access the register save area.  
+     If not, increment the frame pointer as required.  */
 
-  return;
-}
+  if (area_top <= area_bottom)
+    {
+      /* Nothing to restore.  */
+    }
+  else if (frame.frame_size + area_bottom >= 0
+           && frame.frame_size + area_top <= 4096)
+    {
+      /* Area is in range.  */
+      offset = frame.frame_size;
+    }
+  else
+    {
+      rtx insn, frame_off;
 
-/* Output the function epilogue assembly code to the 
-   stdio stream FILE.  The local frame size is passed
-   in LSIZE.  */
+      offset = area_bottom < 0 ? -area_bottom : 0; 
+      frame_off = GEN_INT (frame.frame_size - offset);
 
-void
-s390_function_epilogue (file, lsize)
-     FILE *file;
-     HOST_WIDE_INT lsize;
-{
-/* Register is call clobbered and not used for eh or return.  */
-#define FREE_REG 4
+      if (!CONST_OK_FOR_LETTER_P (INTVAL (frame_off), 'K'))
+	frame_off = force_const_mem (Pmode, frame_off);
 
-  int i;
-  long frame_size;
-  int return_reg = RETURN_REGNUM;
-  int fp, offset;
-  const char *const b64 = TARGET_64BIT ? "g" : "";
+      insn = emit_insn (gen_add2_insn (frame_pointer, frame_off));
+      RTX_FRAME_RELATED_P (insn) = 1;
+    }
 
-  frame_size = STARTING_FRAME_OFFSET + lsize + save_fprs_p () * 64;
-  
-  if (current_function_uses_pic_offset_table)
-    regs_ever_live[PIC_OFFSET_TABLE_REGNUM] = 1;
-  
-  if (leaf_function_flag)
+  /* Restore call saved fprs.  */
+
+  if (TARGET_64BIT)
     {
-      for (i = 6; i < 13 && (regs_ever_live[i] == 0); i++);
+      int i;
 
-      if (s390_nr_constants || regs_ever_live[13] || i != 13)
-	{
-	    fprintf (file, "\tlm%s\t%s,%s,%d(%s)\n", b64, 
-                     reg_names[i], reg_names[13],
-		     UNITS_PER_WORD * i,
-		     reg_names[STACK_POINTER_REGNUM]);
-	}
-      if (!TARGET_64BIT)
-	{
-	  if (regs_ever_live[18])
-	    fprintf (file, "\tld\t%s,%d(%s)\n", 
-		     reg_names[18],
-		     STACK_POINTER_OFFSET-16,
-		     reg_names[STACK_POINTER_REGNUM]);
-	  if (regs_ever_live[19])
-	    fprintf (file, "\tld\t%s,%d(%s)\n",
-		     reg_names[19],
-		     STACK_POINTER_OFFSET-8,
-		     reg_names[STACK_POINTER_REGNUM]);
-	}
+      if (frame.save_fprs_p)
+	for (i = 24; i < 32; i++)
+	  if (regs_ever_live[i])
+	    restore_fpr (frame_pointer, 
+			 offset - 64 + (i-24) * 8, i);
     }
   else
     {
-      for (i = 6; i < 13 && (regs_ever_live[i] == 0); i++);
+      if (regs_ever_live[18])
+	restore_fpr (frame_pointer, offset + STACK_POINTER_OFFSET - 16, 18);
+      if (regs_ever_live[19]) 
+	restore_fpr (frame_pointer, offset + STACK_POINTER_OFFSET - 8, 19);
+    }
+
+  /* Return register.  */
+
+  return_reg = gen_rtx_REG (Pmode, RETURN_REGNUM); 
+
+  /* Restore call saved gprs.  */
+
+  if (frame.first_restore_gpr != -1)
+    {
+      rtx addr;
+
+      /* Fetch return address from stack before load multiple,
+	 this will do good for scheduling.  */
 
-      if (frame_size + STACK_POINTER_OFFSET > 4095)    
+      if (frame.last_save_gpr >= RETURN_REGNUM 
+	  && frame.first_restore_gpr < RETURN_REGNUM)
 	{
-	  offset = 0;
-	  fp = STACK_POINTER_REGNUM;
+	  int return_regnum = find_unused_clobbered_reg();
+	  if (!return_regnum)
+	    return_regnum = 4;
+	  return_reg = gen_rtx_REG (Pmode, return_regnum);
+	
+	  addr = plus_constant (frame_pointer, 
+				offset + RETURN_REGNUM * UNITS_PER_WORD); 
+	  addr = gen_rtx_MEM (Pmode, addr);
+	  set_mem_alias_set (addr, s390_sr_alias_set);
+	  emit_move_insn (return_reg, addr);
 	}
-      else if (frame_pointer_needed || current_function_calls_alloca)
+
+      /* ??? As references to the base register are not made
+	 explicit in insn RTX code, we have to add a barrier here
+	 to prevent incorrect scheduling.  */
+
+      emit_insn (gen_blockage());      
+
+      addr = plus_constant (frame_pointer, 
+			    offset + frame.first_restore_gpr * UNITS_PER_WORD);
+      addr = gen_rtx_MEM (Pmode, addr);
+      set_mem_alias_set (addr, s390_sr_alias_set);
+
+      if (frame.first_restore_gpr != frame.last_save_gpr)
 	{
-	  offset = frame_size;
-	  fp = FRAME_POINTER_REGNUM;
+	  emit_insn (gen_load_multiple (
+		       gen_rtx_REG (Pmode, frame.first_restore_gpr),
+                       addr,
+		       GEN_INT (frame.last_save_gpr - frame.first_restore_gpr + 1)));
 	}
       else
 	{
-	  offset = frame_size;
-	  fp = STACK_POINTER_REGNUM;
+	  emit_move_insn (gen_rtx_REG (Pmode, frame.first_restore_gpr),
+		 	  addr); 
 	}
+    }
 
-      /* Restore from offset below save area.  */
+  /* Return to caller.  */
 
-      if (offset == 0)
-	fprintf (file, "\tl%s\t%s,0(%s)\n", b64, 
-		       reg_names[fp], reg_names[fp]);
-      restore_fprs (file, offset-64, fp);
-      return_reg = FREE_REG;
-      fprintf (file, "\tl%s\t%s,%d(%s)\n", b64, reg_names[return_reg], 
-	       UNITS_PER_WORD*RETURN_REGNUM+offset, reg_names[fp]);
-      if (!TARGET_64BIT)
-	{
-	  if (regs_ever_live[18])
-	    fprintf (file, "\tld\t%s,%d(%s)\n", 
-		     reg_names[18],
-		     offset+STACK_POINTER_OFFSET-16, reg_names[fp]);
-	  if (regs_ever_live[19])
-	    fprintf (file, "\tld\t%s,%d(%s)\n",
-		     reg_names[19],
-		     offset+STACK_POINTER_OFFSET-8, reg_names[fp]);
-	}
-      fprintf (file, "\tlm%s\t%s,%s,%d(%s)\n", b64, 
-		     reg_names[i], reg_names[15],
-	             (UNITS_PER_WORD * i) + offset, reg_names[fp]);
-    }
+  p = rtvec_alloc (2);
   
-  fprintf (file, "\tbr\t%s\n", reg_names[return_reg]);
-
-  current_function_uses_pic_offset_table = 0;
-  leaf_function_flag = 0;
-  s390_pool_start_insn = NULL_RTX;
-  s390_pool_count = -1;
-  s390_function_count++;
-  return;
+  RTVEC_ELT (p, 0) = gen_rtx_RETURN (VOIDmode);
+  RTVEC_ELT (p, 1) = gen_rtx_USE (VOIDmode, return_reg);
+  emit_jump_insn (gen_rtx_PARALLEL (VOIDmode, p));
 }
 
 
@@ -3369,3 +3626,98 @@ s390_initialize_trampoline (addr, fnaddr, cxt)
 		   memory_address (Pmode, 
 		   plus_constant (addr, (TARGET_64BIT ? 28 : 16) ))), fnaddr);
 }
+
+/* Return rtx for 64-bit constant formed from the 32-bit subwords
+   LOW and HIGH, independent of the host word size.  */
+
+rtx
+s390_gen_rtx_const_DI (high, low)
+     int high;
+     int low;
+{
+#if HOST_BITS_PER_WIDE_INT >= 64
+  HOST_WIDE_INT val;
+  val = (HOST_WIDE_INT)high;
+  val <<= 32;
+  val |= (HOST_WIDE_INT)low;
+  
+  return GEN_INT (val);
+#else
+#if HOST_BITS_PER_WIDE_INT >= 32
+  return immed_double_const ((HOST_WIDE_INT)low, (HOST_WIDE_INT)high, DImode);
+#else
+  abort ();
+#endif
+#endif
+} 
+
+/* Output assembler code to FILE to increment profiler label # LABELNO
+   for profiling a function entry.  */
+
+void
+s390_function_profiler (file, labelno)
+     FILE *file;
+     int labelno;
+{
+  rtx op[7];
+
+  char label[128];
+  sprintf (label, "%sP%d", LPREFIX, labelno);
+
+  fprintf (file, "# function profiler \n");
+
+  op[0] = gen_rtx_REG (Pmode, RETURN_REGNUM);
+  op[1] = gen_rtx_REG (Pmode, STACK_POINTER_REGNUM);
+  op[1] = gen_rtx_MEM (Pmode, plus_constant (op[1], UNITS_PER_WORD));
+
+  op[2] = gen_rtx_REG (Pmode, 1);
+  op[3] = gen_rtx_SYMBOL_REF (Pmode, label);
+  SYMBOL_REF_FLAG (op[3]) = 1;
+
+  op[4] = gen_rtx_SYMBOL_REF (Pmode, "_mcount");
+  if (flag_pic)
+    {
+      op[4] = gen_rtx_UNSPEC (Pmode, gen_rtvec (1, op[4]), 113);
+      op[4] = gen_rtx_CONST (Pmode, op[4]);
+    }
+
+  if (TARGET_64BIT)
+    {
+      output_asm_insn ("stg\t%0,%1", op);
+      output_asm_insn ("larl\t%2,%3", op);
+      output_asm_insn ("brasl\t%0,%4", op);
+      output_asm_insn ("lg\t%0,%1", op);
+    }
+  else if (!flag_pic)
+    {
+      op[6] = gen_label_rtx ();
+
+      output_asm_insn ("st\t%0,%1", op);
+      output_asm_insn ("bras\t%2,%l6", op);
+      output_asm_insn (".long\t%3", op);
+      output_asm_insn (".long\t%4", op);
+      ASM_OUTPUT_INTERNAL_LABEL (file, "L", CODE_LABEL_NUMBER (op[6]));
+      output_asm_insn ("l\t%0,0(%2)", op);
+      output_asm_insn ("l\t%2,4(%2)", op);
+      output_asm_insn ("basr\t%0,%0", op);
+      output_asm_insn ("l\t%0,%1", op);
+    }
+  else
+    {
+      op[5] = gen_label_rtx ();
+      op[6] = gen_label_rtx ();
+
+      output_asm_insn ("st\t%0,%1", op);
+      output_asm_insn ("bras\t%2,%l6", op);
+      ASM_OUTPUT_INTERNAL_LABEL (file, "L", CODE_LABEL_NUMBER (op[5]));
+      output_asm_insn (".long\t%3-%l5", op);
+      output_asm_insn (".long\t%4-%l5", op);
+      ASM_OUTPUT_INTERNAL_LABEL (file, "L", CODE_LABEL_NUMBER (op[6]));
+      output_asm_insn ("lr\t%0,%2", op);
+      output_asm_insn ("a\t%0,0(%2)", op);
+      output_asm_insn ("a\t%2,4(%2)", op);
+      output_asm_insn ("basr\t%0,%0", op);
+      output_asm_insn ("l\t%0,%1", op);
+    }
+}
+
diff --git a/gcc/config/s390/s390.h b/gcc/config/s390/s390.h
index e9fa7e4..988d7e2 100644
--- a/gcc/config/s390/s390.h
+++ b/gcc/config/s390/s390.h
@@ -65,6 +65,14 @@ extern int target_flags;
 /* Define this to change the optimizations performed by default.  */
 #define OPTIMIZATION_OPTIONS(LEVEL, SIZE) optimization_options(LEVEL, SIZE)
 
+/* Sometimes certain combinations of command options do not make sense
+   on a particular target machine.  You can define a macro
+   `OVERRIDE_OPTIONS' to take account of this.  This macro, if
+   defined, is executed once just after all the command options have
+   been parsed.  */
+#define OVERRIDE_OPTIONS override_options ()
+
+
 /* Defines for REAL_ARITHMETIC.  */
 #define IEEE_FLOAT 1
 #define TARGET_IBM_FLOAT           0
@@ -236,7 +244,12 @@ if (INTEGRAL_MODE_P (MODE) &&	        	    	\
    G5 and following have 16 IEEE floating point register,
    which get numbers 16-31.  */
 
-#define FIRST_PSEUDO_REGISTER 34
+#define FIRST_PSEUDO_REGISTER 35
+
+/* Number of hardware registers that go into the DWARF-2 unwind info.
+   If not defined, equals FIRST_PSEUDO_REGISTER.  */
+
+#define DWARF_FRAME_REGISTERS 34
 
 /* The following register have a fix usage
    GPR 12: GOT register points to the GOT, setup in prologue,
@@ -260,7 +273,7 @@ if (INTEGRAL_MODE_P (MODE) &&	        	    	\
   0, 0, 0, 0, 					\
   0, 0, 0, 0, 					\
   0, 0, 0, 0, 					\
-  1, 1 }
+  1, 1, 1 }
 
 /* 1 for registers not available across function calls.  These must include
    the FIXED_REGISTERS and also any registers that can be used without being
@@ -273,22 +286,49 @@ if (INTEGRAL_MODE_P (MODE) &&	        	    	\
   1, 1, 0, 0, 					\
   0, 0, 0, 0, 					\
   0, 1, 1, 1,					\
+  1, 1, 1, 1, 					\
+  1, 1, 1, 1, 					\
+  1, 1, 1, 1, 					\
+  1, 1, 1, 1, 					\
+  1, 1, 1 }
+
+/* Like `CALL_USED_REGISTERS' except this macro doesn't require that
+   the entire set of `FIXED_REGISTERS' be included.
+   (`CALL_USED_REGISTERS' must be a superset of `FIXED_REGISTERS').  */
+
+#define CALL_REALLY_USED_REGISTERS		\
+{ 1, 1, 1, 1, 					\
   1, 1, 0, 0, 					\
+  0, 0, 0, 0, 					\
+  0, 0, 0, 0,					\
   1, 1, 1, 1, 					\
   1, 1, 1, 1, 					\
   1, 1, 1, 1, 					\
-  1, 1 }
+  1, 1, 1, 1, 					\
+  1, 1, 1 }
 
-/* If not pic code, gpr 12 can be used.  */
+/* Macro to conditionally modify fixed_regs/call_used_regs.  */
 
 #define CONDITIONAL_REGISTER_USAGE				\
 do								\
   {								\
+    int i;							\
+								\
     if (flag_pic)						\
       {								\
 	fixed_regs[PIC_OFFSET_TABLE_REGNUM] = 1;		\
 	call_used_regs[PIC_OFFSET_TABLE_REGNUM] = 1;		\
       }								\
+    if (TARGET_64BIT)						\
+      {								\
+        for (i = 24; i < 32; i++)				\
+	    call_used_regs[i] = call_really_used_regs[i] = 0;	\
+      }								\
+    else							\
+      {								\
+        for (i = 18; i < 20; i++)				\
+	    call_used_regs[i] = call_really_used_regs[i] = 0;	\
+      }								\
  } while (0)
 
 /* The following register have a special usage
@@ -298,7 +338,8 @@ do								\
 	   with stack- or frame-pointer. 
    GPR 33: Condition code 'register' */
 
-#define FRAME_POINTER_REGNUM 11
+#define HARD_FRAME_POINTER_REGNUM 11
+#define FRAME_POINTER_REGNUM 34
 
 #define ARG_POINTER_REGNUM 32
 
@@ -333,7 +374,8 @@ do								\
 #define HARD_REGNO_MODE_OK(REGNO, MODE)                             \
   (FLOAT_REGNO_P(REGNO)?                                            \
    (GET_MODE_CLASS(MODE) == MODE_FLOAT ||                           \
-    GET_MODE_CLASS(MODE) == MODE_COMPLEX_FLOAT) :                   \
+    GET_MODE_CLASS(MODE) == MODE_COMPLEX_FLOAT ||                   \
+    (MODE) == SImode || (MODE) == DImode) :                         \
    INT_REGNO_P(REGNO)?                                              \
     (HARD_REGNO_NREGS(REGNO, MODE) == 1 || !((REGNO) & 1)) :        \
    CC_REGNO_P(REGNO)?                                               \
@@ -349,6 +391,15 @@ do								\
    (((MODE1) == SFmode || (MODE1) == DFmode)	\
    == ((MODE2) == SFmode || (MODE2) == DFmode))
 
+/* If defined, gives a class of registers that cannot be used as the
+   operand of a SUBREG that changes the mode of the object illegally.  */
+
+#define CLASS_CANNOT_CHANGE_MODE FP_REGS
+
+/* Defines illegal mode changes for CLASS_CANNOT_CHANGE_MODE.  */
+
+#define CLASS_CANNOT_CHANGE_MODE_P(FROM,TO) \
+  (GET_MODE_SIZE (FROM) != GET_MODE_SIZE (TO))
 
 /* Define this macro if references to a symbol must be treated
    differently depending on something about the variable or
@@ -383,15 +434,20 @@ while (0)
 
 #define ELIMINABLE_REGS				        \
 {{ FRAME_POINTER_REGNUM, STACK_POINTER_REGNUM},	        \
+ { FRAME_POINTER_REGNUM, HARD_FRAME_POINTER_REGNUM},    \
  { ARG_POINTER_REGNUM, STACK_POINTER_REGNUM},	        \
- { ARG_POINTER_REGNUM, FRAME_POINTER_REGNUM}}  
+ { ARG_POINTER_REGNUM, HARD_FRAME_POINTER_REGNUM}}  
 
 #define CAN_ELIMINATE(FROM, TO) (1)
 
 #define INITIAL_ELIMINATION_OFFSET(FROM, TO, OFFSET) 			  \
 { if ((FROM) == FRAME_POINTER_REGNUM && (TO) == STACK_POINTER_REGNUM) 	  \
   { (OFFSET) = 0; }     						  \
-  else if ((FROM) == ARG_POINTER_REGNUM && (TO) == FRAME_POINTER_REGNUM)  \
+  else  if ((FROM) == FRAME_POINTER_REGNUM                                \
+	    && (TO) == HARD_FRAME_POINTER_REGNUM)                	  \
+  { (OFFSET) = 0; }     						  \
+  else if ((FROM) == ARG_POINTER_REGNUM                                   \
+            && (TO) == HARD_FRAME_POINTER_REGNUM)                         \
   { (OFFSET) = s390_arg_frame_offset (); }     				  \
   else if ((FROM) == ARG_POINTER_REGNUM && (TO) == STACK_POINTER_REGNUM)  \
   { (OFFSET) = s390_arg_frame_offset (); }     				  \
@@ -433,7 +489,8 @@ while (0)
 enum reg_class
 {
   NO_REGS, ADDR_REGS, GENERAL_REGS,
-  FP_REGS, ALL_REGS, LIM_REG_CLASSES
+  FP_REGS, ADDR_FP_REGS, GENERAL_FP_REGS,
+  ALL_REGS, LIM_REG_CLASSES
 };
 
 #define N_REG_CLASSES (int) LIM_REG_CLASSES
@@ -441,7 +498,8 @@ enum reg_class
 /* Give names of register classes as strings for dump file.  */
 
 #define REG_CLASS_NAMES                                                 \
-{ "NO_REGS","ADDR_REGS", "GENERAL_REGS", "FP_REGS", "ALL_REGS" }
+{ "NO_REGS", "ADDR_REGS", "GENERAL_REGS", 				\
+  "FP_REGS", "ADDR_FP_REGS", "GENERAL_FP_REGS", "ALL_REGS" }
 
 /* Define which registers fit in which classes.  This is an initializer for
    a vector of HARD_REG_SET of length N_REG_CLASSES.
@@ -450,10 +508,12 @@ enum reg_class
 #define REG_CLASS_CONTENTS \
 {				       			\
   { 0x00000000, 0x00000000 },	/* NO_REGS */		\
-  { 0x0000fffe, 0x00000001 },	/* ADDR_REGS */		\
-  { 0x0000ffff, 0x00000001 },	/* GENERAL_REGS */	\
+  { 0x0000fffe, 0x00000005 },	/* ADDR_REGS */		\
+  { 0x0000ffff, 0x00000005 },	/* GENERAL_REGS */	\
   { 0xffff0000, 0x00000000 },	/* FP_REGS */		\
-  { 0xffffffff, 0x00000003 },	/* ALL_REGS */		\
+  { 0xfffffffe, 0x00000005 },	/* ADDR_FP_REGS */	\
+  { 0xffffffff, 0x00000005 },	/* GENERAL_FP_REGS */	\
+  { 0xffffffff, 0x00000007 },	/* ALL_REGS */		\
 }
 
 
@@ -506,15 +566,8 @@ extern enum reg_class regclass_map[];	/* smalled class containing REGNO   */
    but on some machines in some cases it is preferable to use a more
    restrictive class.  */
 
-#define PREFERRED_RELOAD_CLASS(X, CLASS)                                 \
-    (GET_CODE (X) == CONST_DOUBLE ?                                      \
-     (GET_MODE_CLASS (GET_MODE (X)) == MODE_FLOAT ? FP_REGS : ADDR_REGS) :\
-     (GET_CODE (X) == CONST_INT ?                                        \
-     (GET_MODE_CLASS (GET_MODE (X)) == MODE_FLOAT ? FP_REGS : ADDR_REGS) :\
-     GET_CODE (X) == PLUS ||                                            \
-     GET_CODE (X) == LABEL_REF ||                                        \
-     GET_CODE (X) == SYMBOL_REF ||                                       \
-     GET_CODE (X) == CONST ? ADDR_REGS : (CLASS)))
+#define PREFERRED_RELOAD_CLASS(X, CLASS)	\
+	s390_preferred_reload_class ((X), (CLASS))
 
 /* Return the maximum number of consecutive registers needed to represent
    mode MODE in a register of class CLASS.  */
@@ -557,26 +610,21 @@ extern enum reg_class regclass_map[];	/* smalled class containing REGNO   */
 
 /* Stack layout; function entry, exit and calling.  */
 
-/* The current return address is on Offset 56 of the current frame
-   if we are in an leaf_function. Otherwise we have to go one stack
-   back.
-   The return address of anything farther back is accessed normally
-   at an offset of 56 from the frame pointer.
-
-   FIXME: builtin_return_addr does not work correctly in a leaf
-          function, we need to find way to find out, if we
-          are in a leaf function
-  */
-
-#define _RETURN_ADDR_OFFSET (TARGET_64BIT ? 112 : 56)
-
-#define RETURN_ADDR_RTX(count, frame)                                   \
-   gen_rtx (MEM, Pmode,                                                 \
-            memory_address (Pmode,                                      \
-                              plus_constant (                           \
-                              copy_to_reg (gen_rtx (MEM, Pmode,         \
-                              memory_address (Pmode, frame))),          \
-                              _RETURN_ADDR_OFFSET)));
+/* The return address of the current frame is retrieved 
+   from the initial value of register RETURN_REGNUM.
+   For frames farther back, we use the stack slot where
+   the corresponding RETURN_REGNUM register was saved.  */
+
+#define DYNAMIC_CHAIN_ADDRESS(FRAME)						\
+  ((FRAME) != hard_frame_pointer_rtx ? (FRAME) :				\
+   plus_constant (arg_pointer_rtx, -STACK_POINTER_OFFSET))
+     
+#define RETURN_ADDR_RTX(COUNT, FRAME)						\
+  ((COUNT) == 0 ? get_hard_reg_initial_val (Pmode, RETURN_REGNUM) :		\
+   gen_rtx_MEM (Pmode,								\
+                memory_address (Pmode, 						\
+                                plus_constant (DYNAMIC_CHAIN_ADDRESS ((FRAME)),	\
+                                               RETURN_REGNUM * UNITS_PER_WORD))))
 
 /* The following macros will turn on dwarf2 exception hndling
    Other code location for this exception handling are 
@@ -588,6 +636,11 @@ extern enum reg_class regclass_map[];	/* smalled class containing REGNO   */
 
 #define MASK_RETURN_ADDR (GEN_INT (0x7fffffff))
 
+/* The offset from the incoming value of %sp to the top of the stack frame
+   for the current function.  */
+
+#define INCOMING_FRAME_SP_OFFSET STACK_POINTER_OFFSET
+
 /* Location, from where return address to load.  */
 
 #define DWARF_FRAME_RETURN_COLUMN  14
@@ -813,47 +866,7 @@ CUMULATIVE_ARGS;
    for profiling a function entry.  */
 
 #define FUNCTION_PROFILER(FILE, LABELNO) 			\
-do {                                     			\
-  extern rtx s390_profile[];  					\
-  extern int s390_pool_count;     				\
-  static char label[128];                     			\
-  fprintf (FILE, "# function profiler \n");   			\
-  if (TARGET_64BIT) 						\
-    {								\
-      rtx tmp[1];						\
-      output_asm_insn ("stg\t14,8(15)", tmp);			\
-      sprintf (label, "%sP%d", LPREFIX, LABELNO);       	\
-      tmp[0] = gen_rtx_SYMBOL_REF (Pmode, label);		\
-      SYMBOL_REF_FLAG (tmp[0]) = 1;				\
-      output_asm_insn ("larl\t1,%0", tmp);      		\
-      tmp[0] = gen_rtx_SYMBOL_REF (Pmode, "_mcount");	        \
-      if (flag_pic)						\
-        {							\
-          tmp[0] = gen_rtx_UNSPEC (Pmode, gen_rtvec (1, tmp[0]), 113); \
-          tmp[0] = gen_rtx_CONST (Pmode, tmp[0]);		\
-        }							\
-      output_asm_insn ("brasl\t14,%0", tmp);			\
-      output_asm_insn ("lg\t14,8(15)", tmp);			\
-    }								\
-  else								\
-    {  								\
-      output_asm_insn ("l     14,4(15)", s390_profile);		\
-      s390_pool_count = 0;                             		\
-      output_asm_insn ("st    14,4(15)", s390_profile);		\
-      output_asm_insn ("l     14,%4", s390_profile);		\
-      output_asm_insn ("l     1,%9", s390_profile);		\
-      if (flag_pic) 						\
-	{   							\
-	  output_asm_insn ("ar    1,13", s390_profile);   	\
-	  output_asm_insn ("bas   14,0(14,13)", s390_profile); 	\
-	} 							\
-      else 							\
-	{							\
-	  output_asm_insn ("basr  14,14", s390_profile);	\
-	}                  					\
-      output_asm_insn ("l     14,4(15)", s390_profile);		\
-    }                     					\
-} while (0)
+	s390_function_profiler ((FILE), ((LABELNO)))
 
 /* #define PROFILE_BEFORE_PROLOGUE */
 
@@ -875,9 +888,9 @@ do {                                     			\
    reg currently allocated to a suitable hard reg.
    These definitions are NOT overridden anywhere.  */
 
-#define REGNO_OK_FOR_INDEX_P(REGNO)                                     \
-  (((REGNO) > 0 && (REGNO) < 16) || (REGNO) == ARG_POINTER_REGNUM       \
-   /* || (REGNO) == FRAME_POINTER_REGNUM */                                 \
+#define REGNO_OK_FOR_INDEX_P(REGNO)					\
+    (((REGNO) < FIRST_PSEUDO_REGISTER 					\
+     && REGNO_REG_CLASS ((REGNO)) == ADDR_REGS) 			\
     || (reg_renumber[REGNO] > 0 && reg_renumber[REGNO] < 16))
 
 #define REGNO_OK_FOR_BASE_P(REGNO) REGNO_OK_FOR_INDEX_P (REGNO)
@@ -947,11 +960,10 @@ do {                                     			\
  * a pseudo reg.  
  */
 
-#define REG_OK_FOR_INDEX_NONSTRICT_P(X)               			\
-((GET_MODE (X) == Pmode) &&						\
- ((REGNO (X) > 0 && REGNO (X) < 16) ||					\
-  (REGNO (X) == ARG_POINTER_REGNUM) ||					\
-  (REGNO (X) >= FIRST_PSEUDO_REGISTER)))
+#define REG_OK_FOR_INDEX_NONSTRICT_P(X)   	\
+((GET_MODE (X) == Pmode) &&			\
+ ((REGNO (X) >= FIRST_PSEUDO_REGISTER) 		\
+  || REGNO_REG_CLASS (REGNO (X)) == ADDR_REGS))  
 
 /* Nonzero if X is a hard reg that can be used as a base reg or if it is
    a pseudo reg.  */
@@ -1080,10 +1092,6 @@ do {                                     			\
 
 /* #define STORE_FLAG_VALUE -1 */
 
-/* When a prototype says `char' or `short', really pass an `int'.  */
-
-#define PROMOTE_PROTOTYPES 1
-
 /* Don't perform CSE on function addresses.  */
 
 #define NO_FUNCTION_CSE
@@ -1269,13 +1277,14 @@ extern struct rtx_def *s390_compare_op0, *s390_compare_op1;
   "%r8",  "%r9", "%r10", "%r11", "%r12", "%r13", "%r14", "%r15",	\
   "%f0",  "%f2",  "%f4",  "%f6",  "%f1",  "%f3",  "%f5",  "%f7",	\
   "%f8",  "%f10", "%f12", "%f14", "%f9", "%f11", "%f13", "%f15",	\
-  "%ap", "%cc"								\
+  "%ap",  "%cc",  "%fp"							\
 }
 
 /* implicit call of memcpy, not bcopy   */
 
 #define TARGET_MEM_FUNCTIONS
 
+
 /* Print operand X (an rtx) in assembler syntax to file FILE.
    CODE is a letter or dot (`z' in `%z0') or 0 if no letter was specified.
    For `%' followed by punctuation, CODE is the punctuation and X is null.  */
@@ -1287,17 +1296,29 @@ extern struct rtx_def *s390_compare_op0, *s390_compare_op1;
 
 /* Define the codes that are matched by predicates in aux-output.c.  */
 
-#define PREDICATE_CODES                                                 \
-  {"s_operand",       { MEM }},                                          \
-  {"bras_sym_operand",{ SYMBOL_REF, CONST }},                            \
-  {"r_or_s_operand",  { MEM, SUBREG, REG }},                             \
-  {"r_or_im8_operand",  { CONST_INT, SUBREG, REG }},                     \
-  {"r_or_s_or_im8_operand",  { MEM, SUBREG, REG, CONST_INT }},           \
-  {"r_or_x_or_im16_operand", { MEM, SUBREG, REG, CONST_INT }},           \
-  {"const0_operand", { CONST_INT, CONST_DOUBLE }},	                 \
-  {"const1_operand", { CONST_INT, CONST_DOUBLE }},	                 \
-  {"tmxx_operand", { CONST_INT, MEM }},
-
+#define PREDICATE_CODES							\
+  {"s_operand",       { SUBREG, MEM }},					\
+  {"s_imm_operand",   { CONST_INT, CONST_DOUBLE, SUBREG, MEM }},	\
+  {"bras_sym_operand",{ SYMBOL_REF, CONST }},				\
+  {"larl_operand",    { SYMBOL_REF, CONST, CONST_INT, CONST_DOUBLE }},	\
+  {"load_multiple_operation", {PARALLEL}},			        \
+  {"store_multiple_operation", {PARALLEL}},			        \
+  {"const0_operand",  { CONST_INT, CONST_DOUBLE }},
+
+
+/* S/390 constant pool breaks the devices in crtstuff.c to control section
+   in where code resides.  We have to write it as asm code.  */
+#ifndef __s390x__
+#define CRT_CALL_STATIC_FUNCTION(func) \
+  if (0) \
+     func (); /* ... to avoid warnings.  */ \
+  else \
+    asm \
+      ("bras\t%%r2,1f\n\
+0:	.long\t" #func " - 0b\n\
+1:	l\t%%r3,0(%%r2)\n\
+	bas\t%%r14,0(%%r3,%%r2)" : : : "2", "3", "cc", "memory");
+#endif
 
 /* Constant Pool for all symbols operands which are changed with
    force_const_mem during insn generation (expand_insn).  */
@@ -1397,7 +1418,11 @@ extern int s390_nr_constants;
           fputc ('\n', (FILE));						    \
 	}								    \
       else								    \
-        assemble_integer (EXP, GET_MODE_SIZE (MODE), ALIGN, 1);		    \
+	{								    \
+	  assemble_integer (EXP, GET_MODE_SIZE (MODE), ALIGN, 1);	    \
+	  if (GET_MODE_SIZE (MODE) == 1)				    \
+	    ASM_OUTPUT_SKIP ((FILE), 1);				    \
+	}								    \
       break;								    \
 									    \
     default:								    \
diff --git a/gcc/config/s390/s390.md b/gcc/config/s390/s390.md
index ea0edac..a056110 100644
--- a/gcc/config/s390/s390.md
+++ b/gcc/config/s390/s390.md
@@ -43,17 +43,12 @@
 ;;
 ;;   s_operand -- Matches a valid S operand in a RS, SI or SS type instruction.
 ;;
-;;   r_or_s_operand -- Matches a register or a valid S operand in a RS, SI
-;;                     or SS type instruction or a register
-;;
-
-;; Insn type.  Used to default other attribute values.
 
 
 ;; Define an insn type attribute.  This is used in function unit delay
 ;; computations.
 
-(define_attr "type" "integer,load,lr,la,store,imul,lmul,fmul,idiv,ldiv,fdiv,branch,jsr,other,o2,o3"
+(define_attr "type" "none,integer,load,lr,la,lm,stm,cs,vs,store,imul,lmul,fmul,idiv,ldiv,fdiv,branch,jsr,other,o2,o3"
   (const_string "integer"))
 
 ;; Insn are devide in two classes:
@@ -66,6 +61,9 @@
 ;; Generic pipeline function unit.  
 
 (define_function_unit "integer" 1 0
+  (eq_attr "type" "none") 0 0)
+
+(define_function_unit "integer" 1 0
   (eq_attr "type" "integer") 1 1)
 
 (define_function_unit "integer" 1 0
@@ -81,6 +79,18 @@
   (eq_attr "type" "store") 1 1)
 
 (define_function_unit "integer" 1 0
+  (eq_attr "type" "lm") 2 2)
+
+(define_function_unit "integer" 1 0
+  (eq_attr "type" "stm") 2 2)
+
+(define_function_unit "integer" 1 0
+  (eq_attr "type" "cs") 5 5)
+
+(define_function_unit "integer" 1 0
+  (eq_attr "type" "vs") 30 30)
+
+(define_function_unit "integer" 1 0
   (eq_attr "type" "jsr") 5 5)
 
 (define_function_unit "integer" 1 0
@@ -235,74 +245,74 @@
 (define_insn "*cmpdi_tm2"
   [(set (reg 33)
         (compare (zero_extract:DI (match_operand:DI 0 "register_operand" "d")
-	                  (match_operand:DI 1 "const1_operand" "")
-                          (match_operand:DI 2 "immediate_operand"  "I"))
+	                          (match_operand:DI 1 "const_int_operand" "n")
+                                  (match_operand:DI 2 "const_int_operand" "n"))
                  (const_int 0)))]
-  "s390_match_ccmode(insn, CCTmode) &&
-   INTVAL(operands[2]) >= 0 && INTVAL(operands[2]) < 64"
+  "s390_match_ccmode(insn, CCTmode) && TARGET_64BIT
+   && INTVAL (operands[1]) >= 1 && INTVAL (operands[2]) >= 0 
+   && INTVAL (operands[1]) + INTVAL (operands[2]) <= 64
+   && (INTVAL (operands[1]) + INTVAL (operands[2]) - 1) >> 4
+      == INTVAL (operands[2]) >> 4"
   "*
 {
-  if (INTVAL (operands[2]) > 47)
-  {
-    operands[1] = GEN_INT (1 << (63 - INTVAL(operands[2])));
-    return \"tmll\\t%0,%x1\";
-  }
-  else if (INTVAL (operands[2]) > 31)
-  {
-    operands[1] = GEN_INT (1 << (47 - INTVAL(operands[2])));
-    return \"tmlh\\t%0,%x1\";
-  } 
-  else if (INTVAL (operands[2]) > 15)
-  {
-    operands[1] = GEN_INT (1 << (31 - INTVAL(operands[2])));
-    return \"tmhl\\t%0,%x1\";
-  }
-  operands[1] = GEN_INT (1 << (15 - INTVAL(operands[2])));
-  return \"tmhh\\t%0,%x1\";
-}"
-  [(set_attr "op_type" "RX")
-   (set_attr "type"    "integer")])
+  int part = INTVAL (operands[2]) >> 4;
+  int block = (1 << INTVAL (operands[1])) - 1;
+  int shift = 16 - INTVAL (operands[1]) - (INTVAL (operands[2]) & 15);
+
+  operands[2] = GEN_INT (block << shift);
 
+  switch (part)
+    {
+      case 0: return \"tmhh\\t%0,%x2\";
+      case 1: return \"tmhl\\t%0,%x2\";
+      case 2: return \"tmlh\\t%0,%x2\";
+      case 3: return \"tmll\\t%0,%x2\";
+      default: abort ();
+    }
+}"
+  [(set_attr "op_type" "RI")])
 
-(define_insn "*cmpdi_tm"
+(define_insn "*cmpdi_tm_reg"
   [(set (reg 33)
         (compare (and:DI (match_operand:DI 0 "register_operand" "%d")
-                         (match_operand:DI 1 "tmxx_operand"  "Lm"))
+                         (match_operand:DI 1 "immediate_operand" "n"))
                  (const_int 0)))]
-  "s390_match_ccmode(insn, CCTmode)"
+  "s390_match_ccmode(insn, CCTmode) && TARGET_64BIT
+   && s390_single_hi (operands[1], DImode, 0) >= 0"
   "*
 {
-  unsigned HOST_WIDEST_INT i;
-  if (GET_CODE (operands[1]) == MEM && 
-      GET_CODE (XEXP (operands[1],0)) == SYMBOL_REF &&
-      CONSTANT_POOL_ADDRESS_P (XEXP (operands[1],0))) 
-    {
-      operands[1] = get_pool_constant (XEXP (operands[1],0));
-    }
-	
-  i = (unsigned HOST_WIDEST_INT) INTVAL (operands[1]);
+  int part = s390_single_hi (operands[1], DImode, 0);
+  operands[1] = GEN_INT (s390_extract_hi (operands[1], DImode, part));
 
-  if (i >= 0x1000000000000ULL)
-    {
-      operands[1] = GEN_INT (i >> 48);
-      return \"tmhh\\t%0,%x1\";
-    }
-  else if (i > 0x100000000ULL)
+  switch (part)
     {
-      operands[1] = GEN_INT (i >> 32);
-      return \"tmhl\\t%0,%x1\";
+      case 0: return \"tmhh\\t%0,%x1\";
+      case 1: return \"tmhl\\t%0,%x1\";
+      case 2: return \"tmlh\\t%0,%x1\";
+      case 3: return \"tmll\\t%0,%x1\";
+      default: abort ();
     }
-  else if (i >= 0x10000ULL)
-    {
-      operands[1] = GEN_INT (i >> 16);
-      return \"tmlh\\t%0,%x1\";
-    } 
-  else
-    return \"tmll\\t%0,%x1\";
 }"
-  [(set_attr "op_type" "RX")
-   (set_attr "type"    "integer")])
+  [(set_attr "op_type" "RI")])
+
+(define_insn "*cmpdi_tm_mem"
+  [(set (reg 33)
+        (compare (and:DI (match_operand:DI 0 "s_operand" "%Qo")
+                         (match_operand:DI 1 "immediate_operand" "n"))
+                 (const_int 0)))]
+  "s390_match_ccmode(insn, CCTmode) && TARGET_64BIT
+   && s390_single_qi (operands[1], DImode, 0) >= 0"
+  "*
+{
+  int part = s390_single_qi (operands[1], DImode, 0);
+  operands[1] = GEN_INT (s390_extract_qi (operands[1], DImode, part));
 
+  operands[0] = gen_rtx_MEM (QImode, 
+			     plus_constant (XEXP (operands[0], 0), part));
+  return \"tm\\t%0,%b1\";
+}"
+  [(set_attr "op_type" "SI")
+   (set_attr "atype"   "mem")])
 
 (define_insn "*ltgr"
   [(set (reg 33)
@@ -312,8 +322,7 @@
         (match_dup 0))]
   "s390_match_ccmode(insn, CCSmode) && TARGET_64BIT"
   "ltgr\\t%2,%0"
-  [(set_attr "op_type" "RRE")
-   (set_attr "type"    "integer")])
+  [(set_attr "op_type" "RRE")])
 
 (define_insn "*cmpdi_ccs_0_64"
   [(set (reg 33)
@@ -321,8 +330,7 @@
                  (match_operand:DI 1 "const0_operand" "")))]
   "s390_match_ccmode(insn, CCSmode) && TARGET_64BIT"
   "ltgr\\t%0,%0"
-  [(set_attr "op_type" "RRE")
-   (set_attr "type"    "integer")])
+  [(set_attr "op_type" "RRE")])
 
 (define_insn "*cmpdi_ccs_0_31"
   [(set (reg 33)
@@ -330,8 +338,7 @@
                  (match_operand:DI 1 "const0_operand" "")))]
   "s390_match_ccmode(insn, CCSmode)"
   "srda\\t%0,0"
-  [(set_attr "op_type" "RS")
-   (set_attr "type"    "integer")])
+  [(set_attr "op_type" "RS")])
 
 (define_insn "*cmpdi_ccs"
   [(set (reg 33)
@@ -358,8 +365,8 @@
 
 (define_insn "*cmpdi_ccu_mem"
   [(set (reg 33)
-        (compare (match_operand:DI 0 "s_operand" "oQ")
-                 (match_operand:DI 1 "s_operand" "oQ")))]
+        (compare (match_operand:DI 0 "s_imm_operand" "oQ")
+                 (match_operand:DI 1 "s_imm_operand" "oQ")))]
   "s390_match_ccmode(insn, CCUmode)"
   "clc\\t%O0(8,%R0),%1"
   [(set_attr "op_type" "SS")
@@ -367,51 +374,73 @@
 
 ; SI instructions
 
-(define_insn "*cmpsi_cct"
+(define_insn "*cmpsi_tm2"
   [(set (reg 33)
         (compare (zero_extract:SI (match_operand:SI 0 "register_operand" "d")
-	                  (match_operand:SI 1 "const1_operand" "")
-                          (match_operand:SI 2 "immediate_operand"  "I"))
+	                          (match_operand:SI 1 "const_int_operand" "n")
+                                  (match_operand:SI 2 "const_int_operand" "n"))
                  (const_int 0)))]
-  "s390_match_ccmode(insn, CCTmode) &&
-   INTVAL(operands[2]) >= 0 && INTVAL(operands[2]) < 32"
+  "s390_match_ccmode(insn, CCTmode)
+   && INTVAL (operands[1]) >= 1 && INTVAL (operands[2]) >= 0 
+   && INTVAL (operands[1]) + INTVAL (operands[2]) <= 32
+   && (INTVAL (operands[1]) + INTVAL (operands[2]) - 1) >> 4
+      == INTVAL (operands[2]) >> 4"
   "*
 {
-  if (INTVAL (operands[2]) > 15)
+  int part = INTVAL (operands[2]) >> 4;
+  int block = (1 << INTVAL (operands[1])) - 1;
+  int shift = 16 - INTVAL (operands[1]) - (INTVAL (operands[2]) & 15);
+
+  operands[2] = GEN_INT (block << shift);
+
+  switch (part)
     {
-      operands[1] = GEN_INT (1 << (31 - INTVAL(operands[2])));
-      return \"tml\\t%0,%x1\";
+      case 0: return \"tmh\\t%0,%x2\";
+      case 1: return \"tml\\t%0,%x2\";
+      default: abort ();
     }
-  operands[1] = GEN_INT (1 << (15 - INTVAL(operands[2])));
-  return \"tmh\\t%0,%x1\";
 }"
   [(set_attr "op_type" "RI")])
 
-(define_insn "*cmpsi_tm"
+(define_insn "*cmpsi_tm_reg"
   [(set (reg 33)
         (compare (and:SI (match_operand:SI 0 "register_operand" "%d")
-                         (match_operand:SI 1 "tmxx_operand"  "Lm"))
+                         (match_operand:SI 1 "immediate_operand" "n"))
                  (const_int 0)))]
-  "s390_match_ccmode(insn, CCTmode)"
+  "s390_match_ccmode(insn, CCTmode)
+   && s390_single_hi (operands[1], SImode, 0) >= 0"
   "*
 {
-  unsigned long i;
-  if (GET_CODE (operands[1]) == MEM && 
-      GET_CODE (XEXP (operands[1],0)) == SYMBOL_REF &&
-      CONSTANT_POOL_ADDRESS_P (XEXP (operands[1],0))) 
-    {
-      operands[1] = get_pool_constant (XEXP (operands[1],0));
-    }
+  int part = s390_single_hi (operands[1], SImode, 0);
+  operands[1] = GEN_INT (s390_extract_hi (operands[1], SImode, part));
 
-  i = (unsigned long) INTVAL (operands[1]);
-  if (i > 0xffff)
+  switch (part)
     {
-      operands[1] = GEN_INT (i / 0x10000);
-      return \"tmh\\t%0,%x1\";
+      case 0: return \"tmh\\t%0,%x1\";
+      case 1: return \"tml\\t%0,%x1\";
+      default: abort ();
     }
-  return \"tml\\t%0,%x1\";
 }"
-  [(set_attr "op_type" "RX")])
+  [(set_attr "op_type" "RI")])
+
+(define_insn "*cmpsi_tm_mem"
+  [(set (reg 33)
+        (compare (and:SI (match_operand:SI 0 "s_operand" "%Qo")
+                         (match_operand:SI 1 "immediate_operand" "n"))
+                 (const_int 0)))]
+  "s390_match_ccmode(insn, CCTmode)
+   && s390_single_qi (operands[1], SImode, 0) >= 0"
+  "*
+{
+  int part = s390_single_qi (operands[1], SImode, 0);
+  operands[1] = GEN_INT (s390_extract_qi (operands[1], SImode, part));
+
+  operands[0] = gen_rtx_MEM (QImode, 
+			     plus_constant (XEXP (operands[0], 0), part));
+  return \"tm\\t%0,%b1\";
+}"
+  [(set_attr "op_type" "SI")
+   (set_attr "atype"   "mem")])
 
 (define_insn "*ltr"
   [(set (reg 33)
@@ -486,15 +515,35 @@
 
 (define_insn "*cmpsi_ccu_mem"
   [(set (reg 33)
-        (compare (match_operand:SI 0 "s_operand" "oQ")
-                 (match_operand:SI 1 "s_operand" "oQ")))]
+        (compare (match_operand:SI 0 "s_imm_operand" "oQ")
+                 (match_operand:SI 1 "s_imm_operand" "oQ")))]
   "s390_match_ccmode(insn, CCUmode)"
   "clc\\t%O0(4,%R0),%1"
    [(set_attr "op_type" "SS")
     (set_attr "atype"   "mem")])
 
+
 ; HI instructions
 
+(define_insn "*cmphi_tm_sub"
+  [(set (reg 33)
+        (compare (and:SI (subreg:SI (match_operand:HI 0 "s_operand" "%Qo") 0)
+                         (match_operand:SI 1 "immediate_operand" "n"))
+                 (const_int 0)))]
+  "s390_match_ccmode(insn, CCTmode)
+   && s390_single_qi (operands[1], HImode, 0) >= 0"
+  "*
+{
+  int part = s390_single_qi (operands[1], HImode, 0);
+  operands[1] = GEN_INT (s390_extract_qi (operands[1], HImode, part));
+
+  operands[0] = gen_rtx_MEM (QImode, 
+			     plus_constant (XEXP (operands[0], 0), part));
+  return \"tm\\t%0,%b1\";
+}"
+  [(set_attr "op_type" "SI")
+   (set_attr "atype"   "mem")])
+
 (define_insn "*icm3"
   [(set (reg 33)
         (compare (match_operand:HI 0 "s_operand" "Qo")
@@ -527,7 +576,7 @@
 (define_insn "*cmphi_ccu"
   [(set (reg 33)
         (compare (match_operand:HI 0 "register_operand" "d")
-                 (match_operand:HI 1 "s_operand"        "Qo")))]
+                 (match_operand:HI 1 "s_imm_operand" "Qo")))]
   "s390_match_ccmode(insn, CCUmode)"
   "clm\\t%0,3,%1"
   [(set_attr "op_type" "RS")
@@ -535,8 +584,8 @@
 
 (define_insn "*cmphi_ccu_mem"
   [(set (reg 33)
-        (compare (match_operand:HI 0 "s_operand" "oQ")
-                 (match_operand:HI 1 "s_operand" "oQ")))]
+        (compare (match_operand:HI 0 "s_imm_operand" "oQ")
+                 (match_operand:HI 1 "s_imm_operand" "oQ")))]
   "s390_match_ccmode(insn, CCUmode)"
   "clc\\t%O0(2,%R0),%1"
   [(set_attr "op_type" "SS")
@@ -545,6 +594,48 @@
 
 ; QI instructions
 
+(define_insn "*cmpqi_tm2"
+  [(set (reg 33)
+        (compare (zero_extract:SI (match_operand:QI 0 "s_operand" "Qo")
+	                          (match_operand:SI 1 "const_int_operand" "n")
+                                  (match_operand:SI 2 "const_int_operand" "n"))
+                 (const_int 0)))]
+  "s390_match_ccmode(insn, CCTmode)
+   && INTVAL (operands[1]) >= 1 && INTVAL (operands[2]) >= 0 
+   && INTVAL (operands[1]) + INTVAL (operands[2]) <= 8"
+  "*
+{
+  int block = (1 << INTVAL (operands[1])) - 1;
+  int shift = 8 - INTVAL (operands[1]) - INTVAL (operands[2]);
+
+  operands[2] = GEN_INT (block << shift);
+  return \"tm\\t%0,%b2\";
+}"
+  [(set_attr "op_type" "SI")
+   (set_attr "atype"   "mem")])
+
+(define_insn "*cmpqi_tm"
+  [(set (reg 33)
+        (compare (and:QI (match_operand:QI 0 "nonimmediate_operand" "%d,Q")
+                         (match_operand:QI 1 "immediate_operand" "n,n"))
+                 (const_int 0)))]
+  "s390_match_ccmode(insn, CCTmode)"
+  "@
+   tml\\t%0,%b1
+   tm\\t%0,%b1"
+  [(set_attr "op_type" "RI,SI")
+   (set_attr "atype"   "reg,mem")])
+
+(define_insn "*cmpqi_tm_sub"
+  [(set (reg 33)
+        (compare (and:SI (subreg:SI (match_operand:QI 0 "s_operand" "%Qo") 0)
+                         (match_operand:SI 1 "immediate_operand" "n"))
+                 (const_int 0)))]
+  "s390_match_ccmode(insn, CCTmode)"
+  "tm\\t%0,%b1"
+  [(set_attr "op_type" "SI")
+   (set_attr "atype"   "mem")])
+
 (define_insn "*icm1"
   [(set (reg 33)
         (compare (match_operand:QI 0 "s_operand" "Qo")
@@ -582,7 +673,8 @@
    (clobber (match_scratch:QI 2 "=d"))]
   "s390_match_ccmode(insn, CCSmode)"
   "icm\\t%2,1,%0"
-  [(set_attr "op_type" "RS")])
+  [(set_attr "op_type" "RS")
+   (set_attr "atype"   "mem")])
 
 (define_insn "*cmpqi_ccu_0"
   [(set (reg 33)
@@ -596,7 +688,7 @@
 (define_insn "*cmpqi_ccu"
   [(set (reg 33)
         (compare (match_operand:QI 0 "register_operand" "d")
-                 (match_operand:QI 1 "s_operand"        "Qo")))]
+                 (match_operand:QI 1 "s_imm_operand" "Qo")))]
   "s390_match_ccmode(insn, CCUmode)"
   "clm\\t%0,1,%1"
   [(set_attr "op_type" "RS")
@@ -604,8 +696,8 @@
 
 (define_insn "*cmpqi_ccu_immed"
   [(set (reg 33)
-        (compare (match_operand:QI 0 "s_operand"      "Qo")
-                 (match_operand:QI 1 "immediate_operand" "")))]
+        (compare (match_operand:QI 0 "s_operand" "Qo")
+                 (match_operand:QI 1 "const_int_operand" "n")))]
   "s390_match_ccmode(insn, CCUmode) &&
    INTVAL(operands[1]) >= 0 && INTVAL(operands[1]) < 256"
   "cli\\t%0,%1"
@@ -614,8 +706,8 @@
 
 (define_insn "*cmpqi_ccu_mem"
   [(set (reg 33)
-        (compare (match_operand:QI 0 "s_operand" "oQ")
-                 (match_operand:QI 1 "s_operand" "oQ")))]
+        (compare (match_operand:QI 0 "s_imm_operand" "oQ")
+                 (match_operand:QI 1 "s_imm_operand" "oQ")))]
   "s390_match_ccmode(insn, CCUmode)"
   "clc\\t%O0(1,%R0),%1"
   [(set_attr "op_type" "SS")
@@ -643,7 +735,7 @@
 (define_insn "*cmpdf_ccs"
   [(set (reg 33)
         (compare (match_operand:DF 0 "register_operand" "f,f")
-                 (match_operand:DF 1 "nonimmediate_operand" "f,m")))]
+                 (match_operand:DF 1 "general_operand" "f,m")))]
   "s390_match_ccmode(insn, CCSmode) && TARGET_HARD_FLOAT && TARGET_IEEE_FLOAT"
   "@
    cdbr\\t%0,%1
@@ -654,7 +746,7 @@
 (define_insn "*cmpdf_ccs_ibm"
   [(set (reg 33)
         (compare (match_operand:DF 0 "register_operand" "f,f")
-                 (match_operand:DF 1 "nonimmediate_operand" "f,m")))]
+                 (match_operand:DF 1 "general_operand" "f,m")))]
   "s390_match_ccmode(insn, CCSmode) && TARGET_HARD_FLOAT && TARGET_IBM_FLOAT"
   "@
    cdr\\t%0,%1
@@ -684,7 +776,7 @@
 (define_insn "*cmpsf_ccs"
   [(set (reg 33)
         (compare (match_operand:SF 0 "register_operand" "f,f")
-                 (match_operand:SF 1 "nonimmediate_operand" "f,m")))]
+                 (match_operand:SF 1 "general_operand" "f,m")))]
   "s390_match_ccmode(insn, CCSmode) && TARGET_HARD_FLOAT && TARGET_IEEE_FLOAT"
   "@
    cebr\\t%0,%1
@@ -695,7 +787,7 @@
 (define_insn "*cmpsf_ccs"
   [(set (reg 33)
         (compare (match_operand:SF 0 "register_operand" "f,f")
-                 (match_operand:SF 1 "nonimmediate_operand" "f,m")))]
+                 (match_operand:SF 1 "general_operand" "f,m")))]
   "s390_match_ccmode(insn, CCSmode) && TARGET_HARD_FLOAT && TARGET_IBM_FLOAT"
   "@
    cer\\t%0,%1
@@ -712,49 +804,70 @@
 ; movti instruction pattern(s).
 ;
 
+(define_insn "*movti_ss"
+  [(set (match_operand:TI 0 "s_operand" "=Qo")
+        (match_operand:TI 1 "s_imm_operand" "Qo"))]
+  ""
+  "mvc\\t%O0(16,%R0),%1"	
+  [(set_attr "op_type" "SS")
+   (set_attr "atype"   "mem")])
+
 (define_insn "movti"
-  [(set (match_operand:TI 0 "nonimmediate_operand" "=d,d,d,m,Q")
-        (match_operand:TI 1 "general_operand" "d,K,m,d,Q"))]
+  [(set (match_operand:TI 0 "nonimmediate_operand" "=d,Q,d,m")
+        (match_operand:TI 1 "general_operand" "Q,d,dKm,d"))]
   "TARGET_64BIT"
-  "*
+  "@
+   lmg\\t%0,%N0,%1
+   stmg\\t%1,%N1,%0
+   #
+   #"
+  [(set_attr "op_type" "RSE,RSE,NN,NN")
+   (set_attr "atype"   "mem")])
+
+(define_split
+  [(set (match_operand:TI 0 "nonimmediate_operand" "")
+        (match_operand:TI 1 "general_operand" ""))]
+  "TARGET_64BIT && reload_completed
+   && !s_operand (operands[0], VOIDmode)
+   && !s_operand (operands[1], VOIDmode)
+   && (register_operand (operands[0], VOIDmode)
+       || register_operand (operands[1], VOIDmode))
+   && (!register_operand (operands[0], VOIDmode)
+       || !reg_overlap_mentioned_p (operand_subword (operands[0], 0, 0, TImode),
+                                    operands[1])
+       || !reg_overlap_mentioned_p (operand_subword (operands[0], 1, 0, TImode),
+                                    operands[1]))"
+  [(set (match_dup 2) (match_dup 4))
+   (set (match_dup 3) (match_dup 5))]
+  "
 {
-  switch (which_alternative)
+  if (!register_operand (operands[0], VOIDmode)
+      || !reg_overlap_mentioned_p (operand_subword (operands[0], 0, 0, TImode),
+                                   operands[1]))
     {
-      case 0: /* d <- d */
-        if (REGNO (operands[0]) == REGNO (operands[1]) + 1)
-          return \"lgr\\t%M0,%M1\;lgr\\t%0,%1\";
-        else
-          return \"lgr\\t%0,%1\;lgr\\t%M0,%M1\";
-
-      case 1: /* d <- K */
-        if (INTVAL(operands[1]) < 0)
-          return \"lghi\\t%0,-1\;lghi\\t%M0,%h1\";
-        else
-          return \"lghi\\t%0,0\;lghi\\t%M0,%h1\";
-
-      case 2: /* d <- m */
-        if (s_operand (operands[1], GET_MODE (operands[1])))
-          return \"lmg\\t%0,%M0,%1\";
-        else 
-          return \"la\\t%M0,%1\;lmg\\t%0,%M0,0(%M0)\";
-
-      case 3: /* m <- d */
-        if (!s_operand (operands[0], GET_MODE (operands[0])))
-          return \"stg\\t%1,%0\;stg\\t%M1,%M0\";
-        else
-          return \"stmg\\t%1,%M1,%0\";
-
-      case 4: /* m <- m */
-        return \"mvc\\t%O0(16,%R0),%1\";
-
-      default:
-        abort();
+      operands[2] = operand_subword (operands[0], 0, 0, TImode);
+      operands[3] = operand_subword (operands[0], 1, 0, TImode);
+      operands[4] = operand_subword (operands[1], 0, 0, TImode);
+      operands[5] = operand_subword (operands[1], 1, 0, TImode);
     }
-}"
-  [(set_attr "op_type" "NN,NN,RS,RS,SS")
-   (set_attr "atype"   "reg,reg,mem,mem,mem")
-   (set_attr "type"    "o2")
-   (set_attr "length"  "12,8,10,10,*")])
+  else
+    {
+      operands[2] = operand_subword (operands[0], 1, 0, TImode);
+      operands[3] = operand_subword (operands[0], 0, 0, TImode);
+      operands[4] = operand_subword (operands[1], 1, 0, TImode);
+      operands[5] = operand_subword (operands[1], 0, 0, TImode);
+    }
+}")
+
+(define_split
+  [(set (match_operand:TI 0 "register_operand" "")
+        (match_operand:TI 1 "memory_operand" ""))]
+  "TARGET_64BIT && reload_completed
+   && !s_operand (operands[1], VOIDmode)"
+  [(set (match_dup 2) (match_dup 3))
+   (set (match_dup 0) (mem:TI (match_dup 2)))]
+  "operands[2] = operand_subword (operands[0], 0, 0, TImode);
+   operands[3] = XEXP (operands[1], 0);")
 
 ;
 ; movdi instruction pattern(s).
@@ -769,73 +882,147 @@
   ""
   "
 {
-  if (CONSTANT_P (operands[1]) 
-      && !LEGITIMATE_CONSTANT_P (operands[1]))
-    operands[1] = force_const_mem (DImode, operands[1]);
-
+  /* Handle PIC symbolic constants.  */
   if (TARGET_64BIT && flag_pic && SYMBOLIC_CONST (operands[1]))
     emit_pic_move (operands, DImode);
+
+  /* During and after reload, we need to force constants
+     to the literal pool ourselves, if necessary.  */
+  if ((reload_in_progress || reload_completed)
+      && CONSTANT_P (operands[1]) 
+      && (!legitimate_reload_constant_p (operands[1])
+          || fp_operand (operands[0], VOIDmode)))
+    operands[1] = force_const_mem (DImode, operands[1]);
 }")
 
+(define_insn "*movdi_lhi"
+  [(set (match_operand:DI 0 "register_operand" "=d")
+        (match_operand:DI 1 "immediate_operand" "K"))]
+  "TARGET_64BIT
+   && GET_CODE (operands[1]) == CONST_INT
+   && CONST_OK_FOR_LETTER_P (INTVAL (operands[1]), 'K')
+   && !fp_operand (operands[0], VOIDmode)"
+  "lghi\\t%0,%h1"
+  [(set_attr "op_type" "RI")
+   (set_attr "atype"   "reg")])
+
+(define_insn "*movdi_lli"
+  [(set (match_operand:DI 0 "register_operand" "=d")
+        (match_operand:DI 1 "immediate_operand" "n"))]
+  "TARGET_64BIT && s390_single_hi (operands[1], DImode, 0) >= 0
+   && !fp_operand (operands[0], VOIDmode)"
+  "*
+{
+  int part = s390_single_hi (operands[1], DImode, 0);
+  operands[1] = GEN_INT (s390_extract_hi (operands[1], DImode, part));
+
+  switch (part)
+    {
+      case 0: return \"llihh\\t%0,%x1\";
+      case 1: return \"llihl\\t%0,%x1\";
+      case 2: return \"llilh\\t%0,%x1\";
+      case 3: return \"llill\\t%0,%x1\";
+      default: abort ();
+    }
+}"
+  [(set_attr "op_type" "RI")
+   (set_attr "atype"   "reg")])
+
+(define_insn "*movdi_larl"
+  [(set (match_operand:DI 0 "register_operand" "=d")
+        (match_operand:DI 1 "larl_operand" "X"))]
+  "TARGET_64BIT
+   && !fp_operand (operands[0], VOIDmode)"
+  "larl\\t%0,%1"
+   [(set_attr "op_type" "RIL")
+    (set_attr "atype"   "reg")
+    (set_attr "type"    "la")])
+
+(define_insn "*movdi_ss"
+  [(set (match_operand:DI 0 "s_operand" "=Qo")
+        (match_operand:DI 1 "s_imm_operand" "Qo"))]
+  ""
+  "mvc\\t%O0(8,%R0),%1"	
+  [(set_attr "op_type" "SS")
+   (set_attr "atype"   "mem")])
+
 (define_insn "*movdi_64"
-  [(set (match_operand:DI 0 "nonimmediate_operand" "=d,d,d,d,m,Q")
-        (match_operand:DI 1 "general_operand" "d,K,S,m,d,Q"))]
+  [(set (match_operand:DI 0 "nonimmediate_operand" "=d,d,m,f,f,m")
+        (match_operand:DI 1 "general_operand" "d,m,d,f,m,f"))]
   "TARGET_64BIT"
   "@
    lgr\\t%0,%1
-   lghi\\t%0,%h1
-   larl\\t%0,%1
    lg\\t%0,%1
    stg\\t%1,%0
-   mvc\\t%O0(8,%R0),%1"	
-   [(set_attr "op_type" "RRE,RI,RIL,RXE,RXE,SS")
-    (set_attr "type"    "integer,integer,la,integer,integer,integer")
-    (set_attr "atype"   "reg,reg,reg,mem,mem,mem")])
+   ldr\\t%0,%1
+   ld\\t%0,%1
+   std\\t%1,%0"
+  [(set_attr "op_type" "RR,RXE,RXE,RR,RX,RX")
+   (set_attr "atype"   "reg,mem,mem,reg,mem,mem")])
 
 (define_insn "*movdi_31"
-  [(set (match_operand:DI 0 "nonimmediate_operand" "=d,d,d,m,Q")
-        (match_operand:DI 1 "general_operand" "d,K,m,d,Q"))]
+  [(set (match_operand:DI 0 "nonimmediate_operand" "=d,Q,d,m,f,f,m")
+        (match_operand:DI 1 "general_operand" "Q,d,dKm,d,f,m,f"))]
   "!TARGET_64BIT"
-  "*
+  "@
+   lm\\t%0,%N0,%1
+   stm\\t%1,%N1,%0
+   #
+   #
+   ldr\\t%0,%1
+   ld\\t%0,%1
+   std\\t%1,%0"
+  [(set_attr "op_type" "RS,RS,NN,NN,RR,RX,RX")
+   (set_attr "atype"   "mem,mem,*,*,reg,mem,mem")])
+
+(define_split
+  [(set (match_operand:DI 0 "nonimmediate_operand" "")
+        (match_operand:DI 1 "general_operand" ""))]
+  "!TARGET_64BIT && reload_completed
+   && !fp_operand (operands[0], VOIDmode)
+   && !fp_operand (operands[1], VOIDmode)
+   && !s_operand (operands[0], VOIDmode)
+   && !s_operand (operands[1], VOIDmode)
+   && (register_operand (operands[0], VOIDmode)
+       || register_operand (operands[1], VOIDmode))
+   && (!register_operand (operands[0], VOIDmode)
+       || !reg_overlap_mentioned_p (operand_subword (operands[0], 0, 0, DImode),
+                                    operands[1])
+       || !reg_overlap_mentioned_p (operand_subword (operands[0], 1, 0, DImode),
+                                    operands[1]))"
+  [(set (match_dup 2) (match_dup 4))
+   (set (match_dup 3) (match_dup 5))]
+  "
 {
-  switch (which_alternative)
+  if (!register_operand (operands[0], VOIDmode)
+      || !reg_overlap_mentioned_p (operand_subword (operands[0], 0, 0, DImode),
+                                   operands[1]))
     {
-      case 0: /* d <- d */
-        if (REGNO (operands[0]) == REGNO (operands[1]) + 1)
-          return \"lr\\t%N0,%N1\;lr\\t%0,%1\";
-        else
-          return \"lr\\t%0,%1\;lr\\t%N0,%N1\";
-
-      case 1: /* d <- K */
-        if (INTVAL (operands[1]) < 0)
-          return \"lhi\\t%0,-1\;lhi\\t%N0,%h1\";
-        else
-          return \"lhi\\t%0,0\;lhi\\t%N0,%h1\";
-
-      case 2: /* d <- m */
-        if (s_operand (operands[1], GET_MODE (operands[1])))
-          return \"lm\\t%0,%N0,%1\";
-        else
-          return \"la\\t%N0,%1\;lm\\t%0,%N0,0(%N0)\";
-
-      case 3: /* m <- d */
-        if (s_operand (operands[0], GET_MODE (operands[0])))
-          return \"stm\\t%1,%N1,%0\";
-        else
-          return \"st\\t%1,%0\;st\\t%N1,%N0\";
-
-      case 4: /* m <- m */
-        return \"mvc\\t%O0(8,%R0),%1\";
-
-      default:
-        abort();
-     }
-}"
-  [(set_attr "op_type" "NN,NN,RS,RS,SS")
-   (set_attr "atype"   "reg,reg,mem,mem,mem")
-   (set_attr "type"    "o2")
-   (set_attr "length"  "4,8,8,8,*")])
+      operands[2] = operand_subword (operands[0], 0, 0, DImode);
+      operands[3] = operand_subword (operands[0], 1, 0, DImode);
+      operands[4] = operand_subword (operands[1], 0, 0, DImode);
+      operands[5] = operand_subword (operands[1], 1, 0, DImode);
+    }
+  else
+    {
+      operands[2] = operand_subword (operands[0], 1, 0, DImode);
+      operands[3] = operand_subword (operands[0], 0, 0, DImode);
+      operands[4] = operand_subword (operands[1], 1, 0, DImode);
+      operands[5] = operand_subword (operands[1], 0, 0, DImode);
+    }
+}")
 
+(define_split
+  [(set (match_operand:DI 0 "register_operand" "")
+        (match_operand:DI 1 "memory_operand" ""))]
+  "!TARGET_64BIT && reload_completed
+   && !fp_operand (operands[0], VOIDmode)
+   && !fp_operand (operands[1], VOIDmode)
+   && !s_operand (operands[1], VOIDmode)"
+  [(set (match_dup 2) (match_dup 3))
+   (set (match_dup 0) (mem:DI (match_dup 2)))]
+  "operands[2] = operand_subword (operands[0], 1, 0, DImode);
+   operands[3] = XEXP (operands[1], 0);")
 
 ;
 ; movsi instruction pattern(s).
@@ -850,11 +1037,8 @@
   ""
   "
 {
-  if (CONSTANT_P (operands[1]) 
-      && !LEGITIMATE_CONSTANT_P (operands[1]))
-    operands[1] = force_const_mem (SImode, operands[1]);
-
-  if (flag_pic && SYMBOLIC_CONST (operands[1]))
+  /* Handle PIC symbolic constants.  */
+  if (!TARGET_64BIT && flag_pic && SYMBOLIC_CONST (operands[1]))
     emit_pic_move (operands, SImode);
 
   /* expr.c tries to load an effective address using 
@@ -869,21 +1053,65 @@
       if (operands[1] == operands[0])
         DONE;
     }
+
+  /* During and after reload, we need to force constants
+     to the literal pool ourselves, if necessary.  */
+  if ((reload_in_progress || reload_completed)
+      && CONSTANT_P (operands[1]) 
+      && (!legitimate_reload_constant_p (operands[1])
+          || fp_operand (operands[0], VOIDmode)))
+    operands[1] = force_const_mem (SImode, operands[1]);
 }")
 
+(define_insn "*movsi_lhi"
+  [(set (match_operand:SI 0 "register_operand" "=d")
+        (match_operand:SI 1 "immediate_operand" "K"))]
+  "GET_CODE (operands[1]) == CONST_INT
+   && CONST_OK_FOR_LETTER_P (INTVAL (operands[1]), 'K')
+   && !fp_operand (operands[0], VOIDmode)"
+  "lhi\\t%0,%h1"
+  [(set_attr "op_type" "RI")])
+
+(define_insn "*movsi_lli"
+  [(set (match_operand:SI 0 "register_operand" "=d")
+        (match_operand:SI 1 "immediate_operand" "n"))]
+  "TARGET_64BIT && s390_single_hi (operands[1], SImode, 0) >= 0
+   && !fp_operand (operands[0], VOIDmode)"
+  "*
+{
+  int part = s390_single_hi (operands[1], SImode, 0);
+  operands[1] = GEN_INT (s390_extract_hi (operands[1], SImode, part));
+
+  switch (part)
+    {
+      case 0: return \"llilh\\t%0,%x1\";
+      case 1: return \"llill\\t%0,%x1\";
+      default: abort ();
+    }
+}"
+  [(set_attr "op_type" "RI")])
+
+(define_insn "*movsi_ss"
+  [(set (match_operand:SI 0 "s_operand" "=Qo")
+        (match_operand:SI 1 "s_imm_operand" "Qo"))]
+  ""
+  "mvc\\t%O0(4,%R0),%1"	
+  [(set_attr "op_type" "SS")
+   (set_attr "atype"   "mem")])
+
 (define_insn "*movsi"
-  [(set (match_operand:SI 0 "nonimmediate_operand" "=d,d,d,m,Q")
-        (match_operand:SI 1 "general_operand" "d,K,m,d,Q"))]
+  [(set (match_operand:SI 0 "nonimmediate_operand" "=d,d,m,f,f,m")
+        (match_operand:SI 1 "general_operand" "d,m,d,f,m,f"))]
   ""
   "@
    lr\\t%0,%1
-   lhi\\t%0,%h1
    l\\t%0,%1
    st\\t%1,%0
-   mvc\\t%O0(4,%R0),%1"	
-  [(set_attr "op_type" "RR,RI,RX,RX,SS")
-   (set_attr "type"    "lr,*,load,store,store")	
-   (set_attr "atype"   "reg,reg,mem,mem,mem")])
+   ler\\t%0,%1
+   le\\t%0,%1
+   ste\\t%1,%0"
+  [(set_attr "op_type" "RR,RX,RX,RR,RX,RX")
+   (set_attr "atype"   "reg,mem,mem,reg,mem,mem")])
 
 
 ;
@@ -892,7 +1120,7 @@
 
 (define_insn "movhi"
   [(set (match_operand:HI 0 "nonimmediate_operand" "=d,d,d,m")
-        (match_operand:HI 1 "r_or_x_or_im16_operand" "d,K,m,d"))]
+        (match_operand:HI 1 "general_operand" "d,n,m,d"))]
   ""
   "@
    lr\\t%0,%1
@@ -902,64 +1130,62 @@
   [(set_attr "op_type" "RR,RI,RX,RX")
    (set_attr "atype"   "reg,reg,mem,mem")])
 
+
 ;
 ; movqi instruction pattern(s).
 ;
 
 (define_insn "movqi_64"
   [(set (match_operand:QI 0 "nonimmediate_operand" "=d,d,d,m,Q")
-        (match_operand:QI 1 "general_operand" "d,K,m,d,n"))]
+        (match_operand:QI 1 "general_operand" "d,n,m,d,n"))]
   "TARGET_64BIT"
   "@
    lr\\t%0,%1
-   llill\\t%0,%x1
+   lhi\\t%0,%b1
    llgc\\t%0,%1
    stc\\t%1,%0
    mvi\\t%0,%b1"
   [(set_attr "op_type" "RR,RI,RXE,RX,SI")
    (set_attr "atype"   "reg,reg,mem,mem,mem")])
 
+
 (define_insn "movqi"
   [(set (match_operand:QI 0 "nonimmediate_operand" "=d,d,d,m,Q")
-        (match_operand:QI 1 "r_or_x_or_im16_operand" "d,n,m,d,n"))]
+        (match_operand:QI 1 "general_operand" "d,n,m,d,n"))]
   ""
   "@
    lr\\t%0,%1
-   lhi\\t%0,%c1
+   lhi\\t%0,%b1
    ic\\t%0,%1
    stc\\t%1,%0
    mvi\\t%0,%b1"
-  [(set_attr "op_type" "RR,RX,RX,RX,SI")
+  [(set_attr "op_type" "RR,RI,RX,RX,SI")
    (set_attr "atype"   "reg,reg,mem,mem,mem")])
 
+
 ;
 ; moveqstrictqi instruction pattern(s).
 ;
 
 (define_insn "*movstrictqi"
-  [(set (strict_low_part (match_operand:QI 0 "nonimmediate_operand" "+d,m"))
-                         (match_operand:QI 1 "nonimmediate_operand"  "m,d"))]
+  [(set (strict_low_part (match_operand:QI 0 "register_operand" "+d"))
+                         (match_operand:QI 1 "memory_operand" "m"))]
   ""
-  "@
-   ic\\t%0,%1
-   stc\\t%1,%0"
-  [(set_attr "op_type"  "RX,RX")
-   (set_attr "atype"    "mem,mem")])
-
+  "ic\\t%0,%1"
+  [(set_attr "op_type"  "RX")
+   (set_attr "atype"    "mem")])
 
 ;
 ; movstricthi instruction pattern(s).
 ;
 
 (define_insn "*movstricthi"
-  [(set (strict_low_part (match_operand:HI 0 "r_or_s_operand" "+d,Q"))
-                         (match_operand:HI 1 "r_or_s_operand"  "Q,d"))
+  [(set (strict_low_part (match_operand:HI 0 "register_operand" "+d"))
+                         (match_operand:HI 1 "s_imm_operand" "Qo"))
    (clobber (reg:CC 33))]
   ""
-  "@
-   icm\\t%0,3,%1
-   stcm\\t%1,3,%0"
-  [(set_attr "op_type" "RS,RS")
+  "icm\\t%0,3,%1"
+  [(set_attr "op_type" "RS")
    (set_attr "atype"   "mem")])
 
 
@@ -968,15 +1194,14 @@
 ;
 
 (define_insn "movestrictsi"
-  [(set (strict_low_part (match_operand:SI 0 "nonimmediate_operand" "+d,d,m"))
-                         (match_operand:SI 1 "nonimmediate_operand"  "d,m,d"))]
+  [(set (strict_low_part (match_operand:SI 0 "register_operand" "+d,d"))
+                         (match_operand:SI 1 "general_operand" "d,m"))]
   "TARGET_64BIT"
   "@
    lr\\t%0,%1
-   l\\t%0,%1
-   st\\t%1,%0"
-  [(set_attr "op_type" "RR,RS,RS")
-   (set_attr "atype"   "reg,mem,mem")])
+   l\\t%0,%1"
+  [(set_attr "op_type" "RR,RS")
+   (set_attr "atype"   "reg,mem")])
 
 
 ;
@@ -989,120 +1214,98 @@
   ""
   "
 {
-  if (GET_CODE (operands[1]) == CONST_DOUBLE)
-    operands[1] = force_const_mem (DFmode, operands[1]);	
+  /* During and after reload, we need to force constants
+     to the literal pool ourselves, if necessary.  */
+  if ((reload_in_progress || reload_completed)
+      && CONSTANT_P (operands[1]))
+    operands[1] = force_const_mem (DFmode, operands[1]);
 }")
 
+(define_insn "*movdf_ss"
+  [(set (match_operand:DF 0 "s_operand" "=Qo")
+        (match_operand:DF 1 "s_imm_operand" "Qo"))]
+  ""
+  "mvc\\t%O0(8,%R0),%1"	
+  [(set_attr "op_type" "SS")
+   (set_attr "atype"   "mem")])
+
 (define_insn "*movdf_64"
-  [(set (match_operand:DF 0 "nonimmediate_operand" "=f,f,m,d,m,d,Q")
-        (match_operand:DF 1 "general_operand" "f,m,f,m,d,d,Q"))]
-  "TARGET_64BIT && TARGET_HARD_FLOAT"
+  [(set (match_operand:DF 0 "nonimmediate_operand" "=f,f,m,d,d,m")
+        (match_operand:DF 1 "general_operand" "f,m,f,d,m,d"))]
+  "TARGET_64BIT"
   "@
    ldr\\t%0,%1
    ld\\t%0,%1
    std\\t%1,%0
-   lg\\t%0,%1
-   stg\\t%1,%0
    lgr\\t%0,%1
-   mvc\\t%O0(8,%R0),%1"	
-  [(set_attr "op_type" "RR,RX,RX,RXE,RXE,RR,SS")
-   (set_attr "atype"   "reg,mem,mem,mem,mem,mem,mem")])
+   lg\\t%0,%1
+   stg\\t%1,%0"
+  [(set_attr "op_type" "RR,RX,RX,RR,RXE,RXE")
+   (set_attr "atype"   "reg,mem,mem,reg,mem,mem")])
 
 (define_insn "*movdf_31"
-  [(set (match_operand:DF 0 "nonimmediate_operand" "=f,f,m,d,m,d,Q")
-        (match_operand:DF 1 "general_operand" "f,m,f,m,d,d,Q"))]
-  "TARGET_HARD_FLOAT"
-  "*
-{
-  switch (which_alternative)
-    {
-      case 0: /* f <- f */
-        return \"ldr\\t%0,%1\";
-
-      case 1: /* f <- m */
-        return \"ld\\t%0,%1\";
-
-      case 2: /* m <- f */
-        return \"std\\t%1,%0\";
-
-      case 3: /* d <- m */
-        if (s_operand (operands[1], GET_MODE (operands[1])))
-          return \"lm\\t%0,%N0,%1\";
-        else
-	  return \"la\\t%N0,%1\;lm\\t%0,%N0,0(%N0)\";
-
-      case 4: /* m <- d */
-        if (s_operand (operands[0], GET_MODE (operands[0])))
-          return \"stm\\t%1,%N1,%0\";
-        else
-          return \"st\\t%1,%0\;st\\t%N1,%N0\";
-
-      case 5: /* d <- d */
-        if (REGNO (operands[0]) == REGNO (operands[1]) + 1)
-          return \"lr\\t%N0,%N1\;lr\\t%0,%1\";
-        else
-          return \"lr\\t%0,%1\;lr\\t%N0,%N1\";
-
-      case 6: /* m <- m */
-        return \"mvc\\t%O0(8,%R0),%1\";
-
-      default:
-        abort();
-     }
-}"
-  [(set_attr "op_type" "RR,RX,RX,RS,RS,NN,SS")
-   (set_attr "atype"   "reg,mem,mem,mem,mem,reg,mem")
-   (set_attr "length"  "*,*,*,*,*,4,*")])
-
-(define_insn "*movdf_soft_64"
-  [(set (match_operand:DF 0 "nonimmediate_operand" "=d,m,d,Q")
-        (match_operand:DF 1 "general_operand" "m,d,d,Q"))]
-  "TARGET_64BIT && TARGET_SOFT_FLOAT"
+  [(set (match_operand:DF 0 "nonimmediate_operand" "=f,f,m,d,Q,d,m")
+        (match_operand:DF 1 "general_operand" "f,m,f,Q,d,dKm,d"))]
+  "!TARGET_64BIT"
   "@
-   lg\\t%0,%1
-   stg\\t%1,%0
-   lgr\\t%0,%1
-   mvc\\t%O0(8,%R0),%1"	
-  [(set_attr "op_type" "RXE,RXE,RR,SS")
-   (set_attr "atype"   "mem,mem,mem,mem")])
-
-(define_insn "*movdf_soft_31"
-  [(set (match_operand:DF 0 "nonimmediate_operand" "=!d,d,m,Q")
-        (match_operand:DF 1 "general_operand" "!d,m,d,Q"))]
-  "TARGET_SOFT_FLOAT"
-  "*
+   ldr\\t%0,%1
+   ld\\t%0,%1
+   std\\t%1,%0
+   lm\\t%0,%N0,%1
+   stm\\t%1,%N1,%0
+   #
+   #"
+  [(set_attr "op_type" "RR,RX,RX,RS,RS,NN,NN")
+   (set_attr "atype"   "reg,mem,mem,mem,mem,*,*")])
+
+(define_split
+  [(set (match_operand:DF 0 "nonimmediate_operand" "")
+        (match_operand:DF 1 "general_operand" ""))]
+  "!TARGET_64BIT && reload_completed
+   && !fp_operand (operands[0], VOIDmode)
+   && !fp_operand (operands[1], VOIDmode)
+   && !s_operand (operands[0], VOIDmode)
+   && !s_operand (operands[1], VOIDmode)
+   && (register_operand (operands[0], VOIDmode)
+       || register_operand (operands[1], VOIDmode))
+   && (!register_operand (operands[0], VOIDmode)
+       || !reg_overlap_mentioned_p (operand_subword (operands[0], 0, 0, DFmode),
+                                    operands[1])
+       || !reg_overlap_mentioned_p (operand_subword (operands[0], 1, 0, DFmode),
+                                    operands[1]))"
+  [(set (match_dup 2) (match_dup 4))
+   (set (match_dup 3) (match_dup 5))]
+  "
 {
-  switch (which_alternative) 
+  if (!register_operand (operands[0], VOIDmode)
+      || !reg_overlap_mentioned_p (operand_subword (operands[0], 0, 0, DFmode),
+                                   operands[1]))
     {
-      case 0: /* d <- d */
-        if (REGNO (operands[0]) == REGNO (operands[1]) + 1)
-          return \"lr\\t%N0,%N1\;lr\\t%0,%1\";
-        else
-          return \"lr\\t%0,%1\;lr\\t%N0,%N1\";
-
-      case 1: /* d <- m */
-        if (s_operand (operands[1], GET_MODE (operands[1])))
-          return \"lm\\t%0,%N0,%1\";
-        else
-	  return \"la\\t%N0,%1\;lm\\t%0,%N0,0(%N0)\";
-
-      case 2: /* m <- d */
-        if (s_operand (operands[0], GET_MODE (operands[0])))
-          return \"stm\\t%1,%N1,%0\";
-        else
-          return \"st\\t%1,%0\;st\\t%N1,%N0\";
-
-      case 3: /* m <- m */
-        return \"mvc\\t%O0(8,%R0),%1\";
-
-      default:
-        abort();
+      operands[2] = operand_subword (operands[0], 0, 0, DFmode);
+      operands[3] = operand_subword (operands[0], 1, 0, DFmode);
+      operands[4] = operand_subword (operands[1], 0, 0, DFmode);
+      operands[5] = operand_subword (operands[1], 1, 0, DFmode);
     }
-}"
-  [(set_attr "op_type" "NN,RS,RS,SS")
-   (set_attr "atype"   "reg,mem,mem,mem")
-   (set_attr "length"  "8,*,*,*")])
+  else
+    {
+      operands[2] = operand_subword (operands[0], 1, 0, DFmode);
+      operands[3] = operand_subword (operands[0], 0, 0, DFmode);
+      operands[4] = operand_subword (operands[1], 1, 0, DFmode);
+      operands[5] = operand_subword (operands[1], 0, 0, DFmode);
+    }
+}")
 
+(define_split
+  [(set (match_operand:DF 0 "register_operand" "")
+        (match_operand:DF 1 "memory_operand" ""))]
+  "!TARGET_64BIT && reload_completed
+   && !fp_operand (operands[0], VOIDmode)
+   && !fp_operand (operands[1], VOIDmode)
+   && !s_operand (operands[1], VOIDmode)"
+  [(set (match_dup 2) (match_dup 3))
+   (set (match_dup 0) (mem:DI (match_dup 2)))]
+  "operands[2] = operand_subword (operands[0], 1, 0, DFmode);
+   operands[3] = XEXP (operands[1], 0);")
 
 ;
 ; movsf instruction pattern(s).
@@ -1114,51 +1317,35 @@
   ""
   "
 {
-  if (GET_CODE (operands[1]) == CONST_DOUBLE)
-    operands[1] = force_const_mem (SFmode, operands[1]);	
+  /* During and after reload, we need to force constants
+     to the literal pool ourselves, if necessary.  */
+  if ((reload_in_progress || reload_completed)
+      && CONSTANT_P (operands[1]))
+    operands[1] = force_const_mem (SFmode, operands[1]);
 }")
 
-(define_insn "*movsf_64"
-  [(set (match_operand:SF 0 "nonimmediate_operand" "=f,f,m,d,m,d,Q")
-        (match_operand:SF 1 "general_operand" "f,m,f,m,d,d,Q"))]
-  "TARGET_64BIT && TARGET_HARD_FLOAT"
-  "@
-   ler\\t%0,%1
-   le\\t%0,%1
-   ste\\t%1,%0
-   llgf\\t%0,%1
-   st\\t%1,%0
-   lgr\\t%0,%1
-   mvc\\t%O0(4,%R0),%1"	
-  [(set_attr "op_type" "RR,RX,RX,RXE,RX,RR,SS")
-   (set_attr "atype"   "reg,mem,mem,mem,mem,reg,mem")])
+(define_insn "*movsf_ss"
+  [(set (match_operand:SF 0 "s_operand" "=Qo")
+        (match_operand:SF 1 "s_imm_operand" "Qo"))]
+  ""
+  "mvc\\t%O0(4,%R0),%1"	
+  [(set_attr "op_type" "SS")
+   (set_attr "atype"   "mem")])
 
-(define_insn "*movsf_31"
-  [(set (match_operand:SF 0 "nonimmediate_operand" "=f,f,m,d,m,d,Q")
-        (match_operand:SF 1 "general_operand" "f,m,f,m,d,d,Q"))]
-  "TARGET_HARD_FLOAT"
+(define_insn "*movsf"
+  [(set (match_operand:SF 0 "nonimmediate_operand" "=f,f,m,d,d,m")
+        (match_operand:SF 1 "general_operand" "f,m,f,d,m,d"))]
+  ""
   "@
    ler\\t%0,%1
    le\\t%0,%1
    ste\\t%1,%0
-   l\\t%0,%1
-   st\\t%1,%0
-   lr\\t%0,%1
-   mvc\\t%O0(4,%R0),%1"	
-  [(set_attr "op_type" "RR,RX,RX,RX,RX,RR,SS")
-   (set_attr "atype"   "reg,mem,mem,mem,mem,reg,mem")])
-
-(define_insn "*movsf_soft"
-  [(set (match_operand:SF 0 "nonimmediate_operand" "=d,d,m,Q")
-        (match_operand:SF 1 "general_operand" "d,m,d,Q"))]
-  "TARGET_SOFT_FLOAT"
-  "@
    lr\\t%0,%1
    l\\t%0,%1
-   st\\t%1,%0
-   mvc\\t%O0(4,%R0),%1"	
-  [(set_attr "op_type" "RR,RX,RX,SS")
-   (set_attr "atype"   "reg,mem,mem,mem")])
+   st\\t%1,%0"
+  [(set_attr "op_type" "RR,RX,RX,RR,RX,RX")
+   (set_attr "atype"   "reg,mem,mem,reg,mem,mem")])
+
 ;
 ; load_multiple pattern(s).
 ;
@@ -1173,12 +1360,12 @@
   int regno;
   int count;
   rtx from;
-  int i;
+  int i, off;
 
   /* Support only loading a constant number of fixed-point registers from
      memory and only bother with this if more than two */
   if (GET_CODE (operands[2]) != CONST_INT
-      || INTVAL (operands[2]) <= 2
+      || INTVAL (operands[2]) < 2
       || INTVAL (operands[2]) > 16
       || GET_CODE (operands[1]) != MEM
       || GET_CODE (operands[0]) != REG
@@ -1189,13 +1376,38 @@
   regno = REGNO (operands[0]);
 
   operands[3] = gen_rtx_PARALLEL (VOIDmode, rtvec_alloc (count));
-  from = force_reg (Pmode, XEXP (operands[1], 0));
+  if (no_new_pseudos)
+    {
+      if (GET_CODE (XEXP (operands[1], 0)) == REG)
+	{
+	  from = XEXP (operands[1], 0);
+	  off = 0;
+	}
+      else if (GET_CODE (XEXP (operands[1], 0)) == PLUS
+	       && GET_CODE (XEXP (XEXP (operands[1], 0), 0)) == REG
+	       && GET_CODE (XEXP (XEXP (operands[1], 0), 1)) == CONST_INT)
+	{
+	  from = XEXP (XEXP (operands[1], 0), 0);
+	  off = INTVAL (XEXP (XEXP (operands[1], 0), 1));
+	}
+      else
+	FAIL;
+
+      if (from == frame_pointer_rtx || from == arg_pointer_rtx)
+	FAIL;
+    }
+  else
+    {
+      from = force_reg (Pmode, XEXP (operands[1], 0));
+      off = 0;
+    }
 
   for (i = 0; i < count; i++)
     XVECEXP (operands[3], 0, i)
       = gen_rtx_SET (VOIDmode, gen_rtx_REG (Pmode, regno + i),
 		     change_address (operands[1], Pmode,
-				     plus_constant (from, i * 4)));
+				     plus_constant (from,
+						    off + i * UNITS_PER_WORD)));
 }")
 
 (define_insn "*load_multiple_di"
@@ -1214,7 +1426,8 @@
     return \"lmg\\t%1,%0,%2\";
 }"
    [(set_attr "op_type" "RXE")
-    (set_attr "atype"   "mem")])
+    (set_attr "atype"   "mem")
+    (set_attr "type"    "lm")])
 
 (define_insn "*load_multiple_si"
   [(match_parallel 0 "load_multiple_operation"
@@ -1232,7 +1445,8 @@
     return \"lm\\t%1,%0,%2\";
 }"
    [(set_attr "op_type" "RXE")
-    (set_attr "atype"   "mem")])
+    (set_attr "atype"   "mem")
+    (set_attr "type"    "lm")])
 
 ;
 ; store multiple pattern(s). 
@@ -1248,12 +1462,12 @@
   int regno;
   int count;
   rtx to;
-  int i;
+  int i, off;
 
   /* Support only storing a constant number of fixed-point registers to
      memory and only bother with this if more than two.  */
   if (GET_CODE (operands[2]) != CONST_INT
-      || INTVAL (operands[2]) <= 2
+      || INTVAL (operands[2]) < 2
       || INTVAL (operands[2]) > 16
       || GET_CODE (operands[0]) != MEM
       || GET_CODE (operands[1]) != REG
@@ -1264,13 +1478,39 @@
   regno = REGNO (operands[1]);
 
   operands[3] = gen_rtx_PARALLEL (VOIDmode, rtvec_alloc (count));
-  to = force_reg (Pmode, XEXP (operands[0], 0));
+
+  if (no_new_pseudos)
+    {
+      if (GET_CODE (XEXP (operands[0], 0)) == REG)
+	{
+	  to = XEXP (operands[0], 0);
+	  off = 0;
+	}
+      else if (GET_CODE (XEXP (operands[0], 0)) == PLUS
+	       && GET_CODE (XEXP (XEXP (operands[0], 0), 0)) == REG
+	       && GET_CODE (XEXP (XEXP (operands[0], 0), 1)) == CONST_INT)
+	{
+	  to = XEXP (XEXP (operands[0], 0), 0);
+	  off = INTVAL (XEXP (XEXP (operands[0], 0), 1));
+	}
+      else
+	FAIL;
+
+      if (to == frame_pointer_rtx || to == arg_pointer_rtx)
+	FAIL;
+    }
+  else	
+    {
+      to = force_reg (Pmode, XEXP (operands[0], 0));
+      off = 0;
+    }
 
   for (i = 0; i < count; i++)
     XVECEXP (operands[3], 0, i)
       = gen_rtx_SET (VOIDmode,
 		     change_address (operands[0], Pmode,
-				     plus_constant (to, i * 4)),
+				     plus_constant (to,
+						    off + i * UNITS_PER_WORD)),
 		     gen_rtx_REG (Pmode, regno + i));
 }")
 
@@ -1291,7 +1531,7 @@
 }"
    [(set_attr "op_type" "RXE")
     (set_attr "atype"   "mem")
-    (set_attr "type"    "other")])
+    (set_attr "type"    "stm")])
 
 
 (define_insn "*store_multiple_si"
@@ -1311,7 +1551,7 @@
 }"
    [(set_attr "op_type" "RXE")
     (set_attr "atype"   "mem")
-    (set_attr "type"    "other")])
+    (set_attr "type"    "stm")])
 
 ;;
 ;; String instructions.
@@ -1357,14 +1597,14 @@
 
           /* Load up the address+length pairs.  */
 
-          emit_move_insn (gen_rtx_SUBREG (DImode, reg0, 0), addr0);
-          emit_move_insn (gen_rtx_SUBREG (DImode, reg0, 8), len);
+          emit_move_insn (gen_highpart (DImode, reg0), addr0);
+          emit_move_insn (gen_lowpart (DImode, reg0), len);
 
-          emit_move_insn (gen_rtx_SUBREG (DImode, reg1, 0), addr1);
-          emit_move_insn (gen_rtx_SUBREG (DImode, reg1, 8), len);
+          emit_move_insn (gen_highpart (DImode, reg1), addr1);
+          emit_move_insn (gen_lowpart (DImode, reg1), len);
 
           /* MOVE */
-          emit_insn (gen_movstrdi_64 (reg0, reg1));
+          emit_insn (gen_movstrdi_64 (reg0, reg1, reg0, reg1));
           DONE;
         }
       else
@@ -1382,7 +1622,7 @@
           emit_move_insn (reg0, addr0);
           emit_move_insn (reg1, addr1);
           emit_insn (gen_adddi3 (len, len, constm1_rtx));
-          emit_insn (gen_movstrdix_64 (reg0, reg1, len));
+          emit_insn (gen_movstrdix_64 (reg0, reg1, reg0, reg1, len));
           emit_label (label); 
           DONE;	   
        }	 
@@ -1431,14 +1671,14 @@
 
           /* Load up the address+length pairs.  */
 
-          emit_move_insn (gen_rtx_SUBREG (SImode, reg0, 0), addr0);
-          emit_move_insn (gen_rtx_SUBREG (SImode, reg0, 4), len);
+          emit_move_insn (gen_highpart (SImode, reg0), addr0);
+          emit_move_insn (gen_lowpart (SImode, reg0), len);
 
-          emit_move_insn (gen_rtx_SUBREG (SImode, reg1, 0), addr1);
-          emit_move_insn (gen_rtx_SUBREG (SImode, reg1, 4), len);
+          emit_move_insn (gen_highpart (SImode, reg1), addr1);
+          emit_move_insn (gen_lowpart (SImode, reg1), len);
 
           /* MOVE */
-          emit_insn (gen_movstrsi_31 (reg0, reg1));
+          emit_insn (gen_movstrsi_31 (reg0, reg1, reg0, reg1));
           DONE;
         }
       else
@@ -1456,7 +1696,7 @@
           emit_move_insn (reg0, addr0);
           emit_move_insn (reg1, addr1);
           emit_insn (gen_addsi3 (len, len, constm1_rtx));
-          emit_insn (gen_movstrsix_31 (reg0, reg1, len));
+          emit_insn (gen_movstrsix_31 (reg0, reg1, reg0, reg1, len));
           emit_label (label); 
           DONE;	   
        }	 
@@ -1477,12 +1717,12 @@
 ; Move a block that is more than 256 bytes in lenght or length in register
 
 (define_insn "movstrdix_64"
-  [(set (mem:BLK (match_operand:DI 0 "register_operand" "a"))
-        (mem:BLK (match_operand:DI 1 "register_operand" "a")))
-   (use (match_operand:DI 2 "register_operand" "a"))
-   (clobber (match_dup 0))
-   (clobber (match_dup 1))
-   (clobber (match_scratch:DI 3 "=&a"))
+  [(clobber (match_operand:DI 0 "register_operand" "=a"))
+   (clobber (match_operand:DI 1 "register_operand" "=a"))
+   (set (mem:BLK (match_operand:DI 2 "register_operand" "0"))
+        (mem:BLK (match_operand:DI 3 "register_operand" "1")))
+   (use (match_operand:DI 4 "register_operand" "a"))
+   (clobber (match_scratch:DI 5 "=&a"))
    (clobber (reg:CC 33))]
   ""
   "*
@@ -1491,36 +1731,34 @@
   xop[0] = gen_label_rtx ();
   xop[1] = gen_label_rtx ();
   xop[2] = gen_label_rtx ();
-  xop[3] = operands[3];
-  output_asm_insn (\"srag\\t%3,%2,8\",operands);
-  output_asm_insn (\"jz\\t%l1\",xop);
+  xop[3] = operands[5];
+  output_asm_insn (\"srag\\t%5,%4,8\", operands);
+  output_asm_insn (\"jz\\t%l1\", xop);
   ASM_OUTPUT_INTERNAL_LABEL (asm_out_file, \"L\",
 			     CODE_LABEL_NUMBER (xop[0]));
-  output_asm_insn (\"mvc\\t0(256,%0),0(%1)\",operands);
-  output_asm_insn (\"la\\t%0,256(%0)\",operands);
-  output_asm_insn (\"la\\t%1,256(%1)\",operands);
-  xop[3] = operands[3];
-  output_asm_insn (\"brct\\t%3,%l0\",xop);
+  output_asm_insn (\"mvc\\t0(256,%0),0(%1)\", operands);
+  output_asm_insn (\"la\\t%0,256(%0)\", operands);
+  output_asm_insn (\"la\\t%1,256(%1)\", operands);
+  output_asm_insn (\"brct\\t%3,%l0\", xop);
   ASM_OUTPUT_INTERNAL_LABEL (asm_out_file, \"L\",
 			     CODE_LABEL_NUMBER (xop[1]));
-  xop[3] = operands[3];
-  output_asm_insn (\"bras\\t%3,%l2\",xop);
-  output_asm_insn (\"mvc\\t0(1,%0),0(%1)\",operands);
+  output_asm_insn (\"bras\\t%3,%l2\", xop);
+  output_asm_insn (\"mvc\\t0(1,%0),0(%1)\", operands);
   ASM_OUTPUT_INTERNAL_LABEL (asm_out_file, \"L\",
 			     CODE_LABEL_NUMBER (xop[2]));
-  return \"ex\\t%2,0(%3)\";
+  return \"ex\\t%4,0(%5)\";
 }"
   [(set_attr "op_type" "NN")
    (set_attr "atype"   "mem")
    (set_attr "length" "44")])
 
 (define_insn "movstrsix_31"
-  [(set (mem:BLK (match_operand:SI 0 "register_operand" "a"))
-        (mem:BLK (match_operand:SI 1 "register_operand" "a")))
-   (use (match_operand:SI 2 "register_operand" "a"))
-   (clobber (match_dup 0))
-   (clobber (match_dup 1))
-   (clobber (match_scratch:SI 3 "=&a"))
+  [(clobber (match_operand:SI 0 "register_operand" "=a"))
+   (clobber (match_operand:SI 1 "register_operand" "=a"))
+   (set (mem:BLK (match_operand:SI 2 "register_operand" "0"))
+        (mem:BLK (match_operand:SI 3 "register_operand" "1")))
+   (use (match_operand:SI 4 "register_operand" "a"))
+   (clobber (match_scratch:SI 5 "=&a"))
    (clobber (reg:CC 33))]
   ""
   "*
@@ -1529,25 +1767,23 @@
   xop[0] = gen_label_rtx ();
   xop[1] = gen_label_rtx ();
   xop[2] = gen_label_rtx ();
-  xop[3] = operands[3];
-  output_asm_insn (\"lr\\t%3,%2\",operands);
-  output_asm_insn (\"sra\\t%3,8\",operands);
-  output_asm_insn (\"jz\\t%l1\",xop);
+  xop[3] = operands[5];
+  output_asm_insn (\"lr\\t%5,%4\", operands);
+  output_asm_insn (\"sra\\t%5,8\", operands);
+  output_asm_insn (\"jz\\t%l1\", xop);
   ASM_OUTPUT_INTERNAL_LABEL (asm_out_file, \"L\",
 			     CODE_LABEL_NUMBER (xop[0]));
-  output_asm_insn (\"mvc\\t0(256,%0),0(%1)\",operands);
-  output_asm_insn (\"la\\t%0,256(%0)\",operands);
-  output_asm_insn (\"la\\t%1,256(%1)\",operands);
-  xop[3] = operands[3];
-  output_asm_insn (\"brct\\t%3,%l0\",xop);
+  output_asm_insn (\"mvc\\t0(256,%0),0(%1)\", operands);
+  output_asm_insn (\"la\\t%0,256(%0)\", operands);
+  output_asm_insn (\"la\\t%1,256(%1)\", operands);
+  output_asm_insn (\"brct\\t%3,%l0\", xop);
   ASM_OUTPUT_INTERNAL_LABEL (asm_out_file, \"L\",
 			     CODE_LABEL_NUMBER (xop[1]));
-  xop[3] = operands[3];
-  output_asm_insn (\"bras\\t%3,%l2\",xop);
-  output_asm_insn (\"mvc\\t0(1,%0),0(%1)\",operands);
+  output_asm_insn (\"bras\\t%3,%l2\", xop);
+  output_asm_insn (\"mvc\\t0(1,%0),0(%1)\", operands);
   ASM_OUTPUT_INTERNAL_LABEL (asm_out_file, \"L\",
 			     CODE_LABEL_NUMBER (xop[2]));
-  return \"ex\\t%2,0(%3)\";
+  return \"ex\\t%4,0(%5)\";
 }"
    [(set_attr "op_type" "NN")
     (set_attr "length" "42")
@@ -1556,10 +1792,10 @@
 ; Move a block that is larger than 255 bytes in length.
 
 (define_insn "movstrdi_64"
-  [(set (mem:BLK (subreg:DI (match_operand:TI 0 "register_operand" "d") 0))
-        (mem:BLK (subreg:DI (match_operand:TI 1 "register_operand" "d") 0)))
-   (clobber (match_dup 0))
-   (clobber (match_dup 1))
+  [(clobber (match_operand:TI 0 "register_operand" "=d"))
+   (clobber (match_operand:TI 1 "register_operand" "=d"))
+   (set (mem:BLK (subreg:DI (match_operand:TI 2 "register_operand" "0") 0))
+        (mem:BLK (subreg:DI (match_operand:TI 3 "register_operand" "1") 0)))
    (clobber (reg:CC 33))]
   ""
   "mvcle\\t%0,%1,0\;jo\\t.-4"
@@ -1568,10 +1804,10 @@
    (set_attr "length"  "8")])
 
 (define_insn "movstrsi_31"
-  [(set (mem:BLK (subreg:SI (match_operand:DI 0 "register_operand" "d") 0))
-        (mem:BLK (subreg:SI (match_operand:DI 1 "register_operand" "d") 0)))
-   (clobber (match_dup 0))
-   (clobber (match_dup 1))
+  [(clobber (match_operand:DI 0 "register_operand" "=d"))
+   (clobber (match_operand:DI 1 "register_operand" "=d"))
+   (set (mem:BLK (subreg:SI (match_operand:DI 2 "register_operand" "0") 0))
+        (mem:BLK (subreg:SI (match_operand:DI 3 "register_operand" "1") 0)))
    (clobber (reg:CC 33))]
   ""
   "mvcle\\t%0,%1,0\;jo\\t.-4"
@@ -1611,13 +1847,13 @@
 
       /* Load up the address+length pairs.  */
 
-      emit_move_insn (gen_rtx_SUBREG (DImode, reg0, 0), addr);
-      emit_move_insn (gen_rtx_SUBREG (DImode, reg0, 8), len);
+      emit_move_insn (gen_highpart (DImode, reg0), addr);
+      emit_move_insn (gen_lowpart (DImode, reg0), len);
 
-      emit_move_insn (gen_rtx_SUBREG (DImode, reg1, 8), const0_rtx);
+      emit_move_insn (gen_lowpart (DImode, reg1), const0_rtx);
  
       /* Clear! */
-      emit_insn (gen_clrstrsi_64 (reg0, reg1));
+      emit_insn (gen_clrstrsi_64 (reg0, reg1, reg0, reg1));
       DONE;	
      }
 }")
@@ -1654,13 +1890,13 @@
 
       /* Load up the address+length pairs.  */
 
-      emit_move_insn (gen_rtx_SUBREG (SImode, reg0, 0), addr);
-      emit_move_insn (gen_rtx_SUBREG (SImode, reg0, 4), len);
+      emit_move_insn (gen_highpart (SImode, reg0), addr);
+      emit_move_insn (gen_lowpart (SImode, reg0), len);
 
-      emit_move_insn (gen_rtx_SUBREG (SImode, reg1, 4), const0_rtx);
+      emit_move_insn (gen_lowpart (SImode, reg1), const0_rtx);
  
       /* CLear! */
-      emit_insn (gen_clrstrsi_31 (reg0, reg1));
+      emit_insn (gen_clrstrsi_31 (reg0, reg1, reg0, reg1));
       DONE;	
      }
 }")
@@ -1675,36 +1911,37 @@
   ""
   "xc\\t%O0(%1,%R0),%0"	
   [(set_attr "op_type" "RS")
+   (set_attr "type"    "cs")
    (set_attr "atype"   "mem")])
 
 ; Clear memory with length greater 256 bytes or lenght not constant
 
 (define_insn "clrstrsi_64"
-  [(set (mem:BLK (subreg:DI (match_operand:TI 0 "register_operand" "d") 0))
+  [(clobber (match_operand:TI 0 "register_operand" "=d"))
+   (clobber (match_operand:TI 1 "register_operand" "=d"))
+   (set (mem:BLK (subreg:DI (match_operand:TI 2 "register_operand" "0") 0))
         (const_int 0))
-   (use (match_operand:TI 1 "register_operand" "d"))
-   (clobber (match_dup 0))
-   (clobber (match_dup 1))
+   (use (match_operand:TI 3 "register_operand" "1"))
    (clobber (reg:CC 33))]
   "TARGET_64BIT"
   "mvcle\\t%0,%1,0\;jo\\t.-4"
   [(set_attr "op_type" "NN")
    (set_attr "atype"   "mem")
-   (set_attr "type"   "other")
+   (set_attr "type"    "vs")
    (set_attr "length"  "8")])
 
 (define_insn "clrstrsi_31"
-  [(set (mem:BLK (subreg:SI (match_operand:DI 0 "register_operand" "d") 0))
+  [(clobber (match_operand:DI 0 "register_operand" "=d"))
+   (clobber (match_operand:DI 1 "register_operand" "=d"))
+   (set (mem:BLK (subreg:SI (match_operand:DI 2 "register_operand" "0") 0))
         (const_int 0))
-   (use (match_operand:DI 1 "register_operand" "d"))
-   (clobber (match_dup 0))
-   (clobber (match_dup 1))
+   (use (match_operand:DI 3 "register_operand" "1"))
    (clobber (reg:CC 33))]
   "!TARGET_64BIT"
   "mvcle\\t%0,%1,0\;jo\\t.-4"
   [(set_attr "op_type" "NN")
    (set_attr "atype"   "mem")
-   (set_attr "type"    "other")
+   (set_attr "type"    "vs")
    (set_attr "length"  "8")])
 
 ;
@@ -1755,14 +1992,14 @@
           len = force_reg (DImode, len);
 
       /* Load up the address+length pairs.  */
-      emit_move_insn (gen_rtx_SUBREG (DImode, reg0, 0), addr0); 
-      emit_move_insn (gen_rtx_SUBREG (DImode, reg0, 8), len);
+      emit_move_insn (gen_highpart (DImode, reg0), addr0); 
+      emit_move_insn (gen_lowpart (DImode, reg0), len);
 
-      emit_move_insn (gen_rtx_SUBREG (DImode, reg1, 0), addr1);
-      emit_move_insn (gen_rtx_SUBREG (DImode, reg1, 8), len);
+      emit_move_insn (gen_highpart (DImode, reg1), addr1);
+      emit_move_insn (gen_lowpart (DImode, reg1), len);
 
       /* Compare! */
-      emit_insn (gen_cmpstr_64 (reg0, reg1));
+      emit_insn (gen_cmpstr_64 (reg0, reg1, reg0, reg1));
       emit_insn (gen_cmpint_di (operands[0]));
       DONE;
     }
@@ -1826,19 +2063,17 @@
           len = force_reg (Pmode, len);
 
       /* Load up the address+length pairs.  */
-      emit_move_insn (gen_rtx_SUBREG (Pmode, reg0, 0), addr0); 
-      emit_move_insn (gen_rtx_SUBREG (Pmode, reg0, 
-	GET_MODE_SIZE (Pmode)), len);
+      emit_move_insn (gen_highpart (Pmode, reg0), addr0); 
+      emit_move_insn (gen_lowpart (Pmode, reg0), len);
 
-      emit_move_insn (gen_rtx_SUBREG (Pmode, reg1, 0), addr1);
-      emit_move_insn (gen_rtx_SUBREG (Pmode, reg1, 
-	GET_MODE_SIZE (Pmode)), len);
+      emit_move_insn (gen_highpart (Pmode, reg1), addr1);
+      emit_move_insn (gen_lowpart (Pmode, reg1), len);
 
       /* Compare! */
       if (TARGET_64BIT) 
-          emit_insn (gen_cmpstr_64 (reg0, reg1));
+          emit_insn (gen_cmpstr_64 (reg0, reg1, reg0, reg1));
       else
-          emit_insn (gen_cmpstr_31 (reg0, reg1));
+          emit_insn (gen_cmpstr_31 (reg0, reg1, reg0, reg1));
 
       emit_insn (gen_cmpint_si (operands[0]));
       DONE;
@@ -1856,37 +2091,33 @@
   "clc\\t%O0(%c2,%R0),%1"
   [(set_attr "op_type" "SS")
    (set_attr "atype"   "mem")
-   (set_attr "type"    "other")])
+   (set_attr "type"    "cs")])
 
 ; Compare a block that is larger than 255 bytes in length.
 
 (define_insn "cmpstr_64"
-  [(set (reg:CCS 33)
-        (compare:CCS (mem:BLK (subreg:DI (match_operand:TI 0 "register_operand" "d") 0))
-                     (mem:BLK (subreg:DI (match_operand:TI 1 "register_operand" "d") 0))))
-   (clobber (subreg:DI (match_dup 0) 0))
-   (clobber (subreg:DI (match_dup 0) 8))
-   (clobber (subreg:DI (match_dup 1) 0))
-   (clobber (subreg:DI (match_dup 1) 8))]
+  [(clobber (match_operand:TI 0 "register_operand" "=d"))
+   (clobber (match_operand:TI 1 "register_operand" "=d"))
+   (set (reg:CCS 33)
+        (compare:CCS (mem:BLK (subreg:DI (match_operand:TI 2 "register_operand" "0") 0))
+                     (mem:BLK (subreg:DI (match_operand:TI 3 "register_operand" "1") 0))))]
   "TARGET_64BIT"
   "clcl\\t%0,%1"
   [(set_attr "op_type" "RR")
    (set_attr "atype"   "mem")
-   (set_attr "type"    "other")])
+   (set_attr "type"    "vs")])
 
 (define_insn "cmpstr_31"
-  [(set (reg:CCS 33)
-        (compare:CCS (mem:BLK (subreg:SI (match_operand:DI 0 "register_operand" "d") 0))
-                     (mem:BLK (subreg:SI (match_operand:DI 1 "register_operand" "d") 0))))
-   (clobber (subreg:SI (match_dup 0) 0))
-   (clobber (subreg:SI (match_dup 0) 4))
-   (clobber (subreg:SI (match_dup 1) 0))
-   (clobber (subreg:SI (match_dup 1) 4))]
+  [(clobber (match_operand:DI 0 "register_operand" "=d"))
+   (clobber (match_operand:DI 1 "register_operand" "=d"))
+   (set (reg:CCS 33)
+        (compare:CCS (mem:BLK (subreg:SI (match_operand:DI 2 "register_operand" "0") 0))
+                     (mem:BLK (subreg:SI (match_operand:DI 3 "register_operand" "1") 0))))]
   "!TARGET_64BIT"
   "clcl\\t%0,%1"
   [(set_attr "op_type" "RR")
    (set_attr "atype"   "mem")
-   (set_attr "type"    "other")])
+   (set_attr "type"    "vs")])
 
 ; Convert condition code to integer in range (-1, 0, 1)
 
@@ -1924,15 +2155,116 @@
    (set_attr "atype"   "reg")
    (set_attr "type"    "other")])
 
+
 ;;
 ;;- Conversion instructions.
 ;;
 
+(define_insn "*sethighqisi"
+  [(set (match_operand:SI 0 "register_operand" "=d")
+        (unspec:SI [(match_operand:QI 1 "s_operand" "Qo")] 10))
+   (clobber (reg:CC 33))]
+  ""
+  "icm\\t%0,8,%1"
+  [(set_attr "op_type" "RS")
+   (set_attr "atype"   "mem")])
+
+(define_insn "*sethighhisi"
+  [(set (match_operand:SI 0 "register_operand" "=d")
+        (unspec:SI [(match_operand:HI 1 "s_operand" "Qo")] 10))
+   (clobber (reg:CC 33))]
+  ""
+  "icm\\t%0,12,%1"
+  [(set_attr "op_type" "RS")
+   (set_attr "atype"   "mem")])
+
+(define_insn "*sethighqidi_64"
+  [(set (match_operand:DI 0 "register_operand" "=d")
+        (unspec:DI [(match_operand:QI 1 "s_operand" "Qo")] 10))
+   (clobber (reg:CC 33))]
+  "TARGET_64BIT"
+  "icmh\\t%0,8,%1"
+  [(set_attr "op_type" "RSE")
+   (set_attr "atype"   "mem")])
+
+(define_insn "*sethighqidi_31"
+  [(set (match_operand:DI 0 "register_operand" "=d")
+        (unspec:DI [(match_operand:QI 1 "s_operand" "Qo")] 10))
+   (clobber (reg:CC 33))]
+  "!TARGET_64BIT"
+  "icm\\t%0,8,%1"
+  [(set_attr "op_type" "RS")
+   (set_attr "atype"   "mem")])
+
+(define_split
+  [(set (match_operand:SI 0 "register_operand" "")
+        (zero_extract:SI (match_operand:QI 1 "s_operand" "")
+                         (match_operand 2 "const_int_operand" "")
+                         (const_int 0)))]
+  "!TARGET_64BIT && !reload_completed
+   && INTVAL (operands[2]) > 0 && INTVAL (operands[2]) < 8"
+  [(parallel
+    [(set (match_dup 0) (unspec:SI [(match_dup 1)] 10))
+     (clobber (reg:CC 33))])
+    (set (match_dup 0) (lshiftrt:SI (match_dup 0) (match_dup 2)))]
+  "
+{
+  operands[2] = GEN_INT (32 - INTVAL (operands[2]));
+  operands[1] = change_address (operands[1], QImode, 0);
+}")
+
+(define_split
+  [(set (match_operand:SI 0 "register_operand" "")
+        (zero_extract:SI (match_operand:QI 1 "s_operand" "")
+                         (match_operand 2 "const_int_operand" "")
+                         (const_int 0)))]
+  "!TARGET_64BIT && !reload_completed
+   && INTVAL (operands[2]) >= 8 && INTVAL (operands[2]) < 16"
+  [(parallel
+    [(set (match_dup 0) (unspec:SI [(match_dup 1)] 10))
+     (clobber (reg:CC 33))])
+    (set (match_dup 0) (lshiftrt:SI (match_dup 0) (match_dup 2)))]
+  "
+{
+  operands[2] = GEN_INT (32 - INTVAL (operands[2]));
+  operands[1] = change_address (operands[1], HImode, 0);
+}")
+
 ;
 ; extendsidi2 instruction pattern(s).
 ;
 
-(define_insn "extendsidi2"
+(define_expand "extendsidi2"
+  [(set (match_operand:DI 0 "register_operand" "")
+        (sign_extend:DI (match_operand:SI 1 "nonimmediate_operand" "")))]
+  ""
+  "
+{
+  if (!TARGET_64BIT)
+    {
+      rtx insns, subword;
+
+      operands[1] = force_reg (SImode, operands[1]);
+      subword = operand_subword (operands[0], 0, 1, DImode);
+
+      start_sequence ();
+      emit_move_insn (subword, operands[1]);
+      emit_insn (gen_ashrdi3 (operands[0], operands[0], GEN_INT (32))); 
+      insns = get_insns ();
+      end_sequence ();
+
+      emit_no_conflict_block (insns, operands[0], operands[1], 0, 
+                              gen_rtx_SIGN_EXTEND (DImode, operands[1]));
+
+      /* Avoid having the REG_RETVAL destroyed by someone attaching 
+         other REG_EQUAL notes.  */
+      emit_move_insn (operands[0], operands[0]);
+      DONE;
+    }
+}
+")
+
+(define_insn "*extendsidi2"
   [(set (match_operand:DI 0 "register_operand" "=d,d")
         (sign_extend:DI (match_operand:SI 1 "nonimmediate_operand" "d,m")))]
   "TARGET_64BIT"
@@ -1942,95 +2274,174 @@
   [(set_attr "op_type" "RRE,RXE")
    (set_attr "atype"   "reg,mem")])
 
-
 ;
 ; extendhidi2 instruction pattern(s).
 ;
 
-(define_insn "extendhidi2"
+(define_expand "extendhidi2"
+  [(set (match_operand:DI 0 "register_operand" "")
+        (sign_extend:DI (match_operand:HI 1 "register_operand" "")))]
+  ""
+  "
+{
+  if (!TARGET_64BIT)
+    {
+      rtx tmp = gen_reg_rtx (SImode);
+      emit_insn (gen_extendhisi2 (tmp, operands[1]));
+      emit_insn (gen_extendsidi2 (operands[0], tmp));
+      DONE;
+    }
+  else
+    {
+      operands[1] = gen_lowpart (DImode, operands[1]);
+      emit_insn (gen_ashldi3 (operands[0], operands[1], GEN_INT (48)));
+      emit_insn (gen_ashrdi3 (operands[0], operands[0], GEN_INT (48))); 
+      DONE;
+    }
+}
+")
+
+(define_insn "*extendhidi2"
   [(set (match_operand:DI 0 "register_operand" "=d")
-        (sign_extend:DI (match_operand:HI 1 "register_operand" "d")))
-   (clobber (reg:CC 33))]
+        (sign_extend:DI (match_operand:HI 1 "memory_operand" "m")))]
   "TARGET_64BIT"
-  "sllg\\t%0,%1,48\;srag\\t%0,%0,48"
-  [(set_attr "op_type" "NN")
-   (set_attr "length"  "12")
-   (set_attr "type"   "o2")])  
+  "lgh\\t%0,%1"
+  [(set_attr "op_type" "RXE")
+   (set_attr "atype"   "mem")])
 
 ;
 ; extendqidi2 instruction pattern(s).
 ;
 
-(define_insn "extendqidi2"
-  [(set (match_operand:DI 0 "register_operand" "=d")
-        (sign_extend:DI (match_operand:QI 1 "register_operand" "d")))
-   (clobber (reg:CC 33))]
-  "TARGET_64BIT"
-  "sllg\\t%0,%1,56\;srag\\t%0,%0,56"
-  [(set_attr "op_type" "NN")
-   (set_attr "length"  "12")
-   (set_attr "type"    "o2")])
+(define_expand "extendqidi2"
+  [(set (match_operand:DI 0 "register_operand" "")
+        (sign_extend:DI (match_operand:QI 1 "register_operand" "")))]
+  ""
+  "
+{
+  if (!TARGET_64BIT)
+    {
+      rtx tmp = gen_reg_rtx (SImode);
+      emit_insn (gen_extendqisi2 (tmp, operands[1]));
+      emit_insn (gen_extendsidi2 (operands[0], tmp));
+      DONE;
+    }
+  else
+    {
+      operands[1] = gen_lowpart (DImode, operands[1]);
+      emit_insn (gen_ashldi3 (operands[0], operands[1], GEN_INT (56)));
+      emit_insn (gen_ashrdi3 (operands[0], operands[0], GEN_INT (56))); 
+      DONE;
+    }
+}
+")
+
+(define_split
+  [(set (match_operand:DI 0 "register_operand" "")
+        (sign_extend:DI (match_operand:QI 1 "s_operand" "")))]
+  "TARGET_64BIT && !reload_completed"
+  [(parallel
+    [(set (match_dup 0) (unspec:DI [(match_dup 1)] 10))
+     (clobber (reg:CC 33))])
+   (parallel
+    [(set (match_dup 0) (ashiftrt:DI (match_dup 0) (const_int 56)))
+     (clobber (reg:CC 33))])]
+  "")
 
 ;
 ; extendhisi2 instruction pattern(s).
 ;
 
-(define_insn "extendhisi2"
-  [(set (match_operand:SI 0 "register_operand" "=d,!d,d")
-        (sign_extend:SI (match_operand:HI 1 "nonimmediate_operand" "0,d,m")))
-   (clobber (reg:CC 33))]
+(define_expand "extendhisi2"
+  [(set (match_operand:SI 0 "register_operand" "")
+        (sign_extend:SI (match_operand:HI 1 "register_operand" "")))]
   ""
-  "@
-   sll\\t%1,16\;sra\\t%1,16
-   lr\\t%0,%1\;sll\\t%0,16\;sra\\t%0,16
-   lh\\t%0,%1"
-  [(set_attr "op_type" "NN,NN,RX")
-   (set_attr "type"   "o2,o3,integer")
-   (set_attr "atype"   "reg,reg,mem")
-   (set_attr "length"  "8,10,*")])
+  "
+{
+  operands[1] = gen_lowpart (SImode, operands[1]);
+  emit_insn (gen_ashlsi3 (operands[0], operands[1], GEN_INT (16)));
+  emit_insn (gen_ashrsi3 (operands[0], operands[0], GEN_INT (16))); 
+  DONE;
+}
+")
 
+(define_insn "*extendhisi2"
+  [(set (match_operand:SI 0 "register_operand" "=d")
+        (sign_extend:SI (match_operand:HI 1 "memory_operand" "m")))]
+  ""
+  "lh\\t%0,%1"
+  [(set_attr "op_type" "RX")
+   (set_attr "atype"   "mem")])
 
 ;
 ; extendqisi2 instruction pattern(s).
 ;
 
-(define_insn "extendqisi2"
-  [(set (match_operand:SI 0 "register_operand" "=d,d")
-        (sign_extend:SI (match_operand:QI 1 "r_or_s_operand" "0,Q")))
-   (clobber (reg:CC 33))]
+(define_expand "extendqisi2"
+  [(set (match_operand:SI 0 "register_operand" "")
+        (sign_extend:SI (match_operand:QI 1 "register_operand" "")))]
   ""
-  "@
-   sll\\t%0,24\;sra\\t%0,24
-   icm\\t%0,8,%1\;sra\\t%0,24"
-  [(set_attr "op_type" "NN,NN")
-   (set_attr "type"    "o2")
-   (set_attr "atype"   "reg,mem")
-   (set_attr "length"  "8,8")])
+  "
+{
+  operands[1] = gen_lowpart (SImode, operands[1]);
+  emit_insn (gen_ashlsi3 (operands[0], operands[1], GEN_INT (24)));
+  emit_insn (gen_ashrsi3 (operands[0], operands[0], GEN_INT (24))); 
+  DONE;
+}
+")
 
+(define_split
+  [(set (match_operand:SI 0 "register_operand" "")
+        (sign_extend:SI (match_operand:QI 1 "s_operand" "")))]
+  "!reload_completed"
+  [(parallel
+    [(set (match_dup 0) (unspec:SI [(match_dup 1)] 10))
+     (clobber (reg:CC 33))])
+   (parallel
+    [(set (match_dup 0) (ashiftrt:SI (match_dup 0) (const_int 24)))
+     (clobber (reg:CC 33))])]
+  "")
 
 ;
 ; extendqihi2 instruction pattern(s).
 ;
 
-(define_insn "extendqihi2"
-  [(set (match_operand:HI 0 "register_operand" "=d,d")
-        (sign_extend:HI (match_operand:QI 1 "r_or_s_operand" "0,Q")))
-   (clobber (reg:CC 33))]
-  ""
-  "@
-   sll\\t%0,24\;sra\\t%0,24
-   icm\\t%0,8,%1\;sra\\t%0,24"
-  [(set_attr "op_type" "NN,NN")
-   (set_attr "type"    "o2")
-   (set_attr "atype"   "reg,mem")
-   (set_attr "length"  "8,8")])
-
 
 ;
 ; zero_extendsidi2 instruction pattern(s).
 ;
 
-(define_insn "zero_extendsidi2"
+(define_expand "zero_extendsidi2"
+  [(set (match_operand:DI 0 "register_operand" "")
+        (zero_extend:DI (match_operand:SI 1 "nonimmediate_operand" "")))]
+  ""
+  "
+{
+  if (!TARGET_64BIT)
+    {
+      rtx insns, subword;
+
+      operands[1] = force_reg (SImode, operands[1]);
+      subword = operand_subword (operands[0], 0, 1, DImode);
+
+      start_sequence ();
+      emit_move_insn (subword, operands[1]);
+      emit_insn (gen_lshrdi3 (operands[0], operands[0], GEN_INT (32))); 
+      insns = get_insns ();
+      end_sequence ();
+
+      emit_no_conflict_block (insns, operands[0], operands[1], 0, 
+                              gen_rtx_ZERO_EXTEND (DImode, operands[1]));
+
+      /* Avoid having the REG_RETVAL destroyed by someone attaching 
+         other REG_EQUAL notes.  */
+      emit_move_insn (operands[0], operands[0]);
+      DONE;
+    }
+}
+")
+
+(define_insn "*zero_extendsidi2"
   [(set (match_operand:DI 0 "register_operand" "=d,d")
         (zero_extend:DI (match_operand:SI 1 "nonimmediate_operand" "d,m")))]
   "TARGET_64BIT"
@@ -2040,288 +2451,148 @@
   [(set_attr "op_type" "RRE,RXE")
    (set_attr "atype"   "reg,mem")])
 
-
 ;
 ; zero_extendhidi2 instruction pattern(s).
 ;
 
-(define_insn "zero_extendhidi2"
-  [(set (match_operand:DI 0 "register_operand" "=!d,d")
-        (zero_extend:DI (match_operand:HI 1 "nonimmediate_operand" "d,m")))]
-  "TARGET_64BIT"
-  "@
-   llgfr\\t%0,%1\;iilh\\t%0,0
-   llgh\\t%0,%1"
-  [(set_attr "op_type" "NN,RXE")
-   (set_attr "type"   "o2,integer")
-   (set_attr "atype"   "reg,mem")
-   (set_attr "length"  "12,*")])	
-
-;
-; zero_extendqidi2 instruction pattern(s)
-;
-
-(define_insn "zero_extendqidi2"
-  [(set (match_operand:DI 0 "register_operand" "=!d,d")
-        (zero_extend:DI (match_operand:QI 1 "nonimmediate_operand" "d,m")))
-   (clobber (reg:CC 33))]
-  "TARGET_64BIT"
-  "@
-   sllg\\t%0,%1,56\;srlg\\t%0,%0,56
-   llgc\\t%0,%1"
-  [(set_attr "op_type" "NN,RXE")
-   (set_attr "type"   "o2,integer")
-   (set_attr "atype"   "reg,mem")
-   (set_attr "length"  "12,*")])
-
-
-;
-; zero_extendhisi2 instruction pattern(s).
-;
-
-(define_expand "zero_extendhisi2"
-  [(set (match_operand:SI 0 "register_operand" "")
-        (zero_extend:SI (match_operand:HI 1 "r_or_s_operand" "")))]
+(define_expand "zero_extendhidi2"
+  [(set (match_operand:DI 0 "register_operand" "")
+        (zero_extend:DI (match_operand:HI 1 "register_operand" "")))]
   ""
   "
 {
   if (!TARGET_64BIT)
     {
-      emit_insn (gen_zero_extendhisi2_31 (operands[0], operands[1],
-		 force_const_mem (SImode, const0_rtx)));
+      rtx tmp = gen_reg_rtx (SImode);
+      emit_insn (gen_zero_extendhisi2 (tmp, operands[1]));
+      emit_insn (gen_zero_extendsidi2 (operands[0], tmp));
       DONE;
     }
-}")
-
-(define_insn "*zero_extendhisi2_64"
-  [(set (match_operand:SI 0 "register_operand" "=d,d")
-        (zero_extend:SI (match_operand:HI 1 "nonimmediate_operand" "0,m")))]
-  "TARGET_64BIT"
-  "@
-   iilh\\t%0,0
-   llgh\\t%0,%1"
-  [(set_attr "op_type" "RI,RXE")
-   (set_attr "atype"   "reg,mem")])
-
-(define_insn "zero_extendhisi2_31"
-  [(set (match_operand:SI 0 "register_operand" "=d,d")
-        (zero_extend:SI (match_operand:HI 1 "r_or_s_operand" "0,Q")))
-   (use (match_operand:SI 2 "memory_operand" "m,m"))
-   (clobber (reg:CC 33))]
-  ""
-  "@
-   icm\\t%0,12,%2
-   icm\\t%0,12,%1\;srl\\t%0,16"
-  [(set_attr "op_type" "RX,NN")
-   (set_attr "type"   "integer,o2")
-   (set_attr "atype"   "reg,mem")
-   (set_attr "length"  "*,8")])
-
-
-;
-; zero_extendqisi2 instruction pattern(s).
-;
-
-(define_insn "*zero_extendqisi2_mem_31"
-  [(set (match_operand:SI 0 "register_operand" "=&d")
-        (zero_extend:SI (match_operand:QI 1 "memory_operand" "m")))
-   (use (match_operand:SI 2 "memory_operand" "m" ))
-   (clobber (reg:CC 33))]
-  ""
-  "sr\\t%0,%0\;ic\\t%0,%1"
-  [(set_attr "op_type" "NN")
-   (set_attr "type"    "o2")
-   (set_attr "atype"   "mem")
-   (set_attr "length"  "6")])
-
-(define_insn "zero_extendqisi2_reg_31"
-  [(set (match_operand:SI 0 "register_operand" "=d")
-        (zero_extend:SI (match_operand:QI 1 "register_operand" "0")))
-   (use (match_operand:SI 2 "memory_operand" "m" ))
-   (clobber (reg:CC 33))]
-  ""
-  "icm\\t%0,14,%2"
-  [(set_attr "op_type" "RX")
-   (set_attr "atype"   "mem")])
-
-(define_insn "*zero_extendqisi2_64"
-  [(set                 (match_operand:SI 0 "register_operand" "=!d,d")
-        (zero_extend:SI (match_operand:QI 1 "nonimmediate_operand"  "d,m")))]
-  "TARGET_64BIT"
-  "@
-   sllg\\t%0,%1,56\;srlg\\t%0,%0,56
-   llgc\\t%0,%1"
-  [(set_attr "op_type" "NN,RXE")
-   (set_attr "type"   "o2,integer")
-   (set_attr "atype"   "reg,mem")
-   (set_attr "length"  "12,*")])
-
-(define_expand "zero_extendqisi2"
-  [(set (match_operand:SI 0 "register_operand" "")
-        (zero_extend:SI (match_operand:QI 1 "r_or_s_operand"  "")))]
-  ""
-  "
-{
-  if (!TARGET_64BIT)
+  else
     {
-      emit_insn (gen_zero_extendqisi2_reg_31 (operands[0], operands[1],
-                              force_const_mem (SImode, const0_rtx)));
+      operands[1] = gen_lowpart (DImode, operands[1]);
+      emit_insn (gen_ashldi3 (operands[0], operands[1], GEN_INT (48)));
+      emit_insn (gen_lshrdi3 (operands[0], operands[0], GEN_INT (48))); 
       DONE;
     }
-}")
+}
+")
 
+(define_insn "*zero_extendhidi2"
+  [(set (match_operand:DI 0 "register_operand" "=d")
+        (zero_extend:DI (match_operand:HI 1 "memory_operand" "m")))]
+  "TARGET_64BIT"
+  "llgh\\t%0,%1"
+  [(set_attr "op_type" "RXE")
+   (set_attr "atype"   "mem")])
 
 ;
-; zero_extendqihi2 instruction pattern(s).
+; zero_extendqidi2 instruction pattern(s)
 ;
 
-(define_insn "zero_extendqihi2_64"
-  [(set (match_operand:HI 0 "register_operand" "=d,d")
-        (zero_extend:HI (match_operand:QI 1 "nonimmediate_operand" "0,m")))
-   (clobber (reg:CC 33))]
-  "TARGET_64BIT"
-  "@
-   nill\\t%0,0x00FF
-   llgc\\t%0,%1"
-  [(set_attr "op_type" "RI,RXE")
-   (set_attr "atype"   "reg,mem")])
-
-(define_insn "zero_extendqihi2_31"
-  [(set (match_operand:HI 0 "register_operand" "=d,&d")
-        (zero_extend:HI (match_operand:QI 1 "nonimmediate_operand" "0,m")))
-   (use (match_operand:SI 2 "memory_operand" "m,m"))
-   (clobber (reg:CC 33))]
-  ""
-  "@
-   icm\\t%0,14,%2
-   sr\\t%0,%0\;ic\\t%0,%1"
-  [(set_attr "op_type" "RX,NN")
-   (set_attr "atype"   "reg,mem")
-   (set_attr "length"  "*,8")])
-
-(define_expand "zero_extendqihi2"
-  [(set (match_operand:HI 0 "register_operand" "")
-        (zero_extend:HI (match_operand:QI 1 "general_operand" "")))]
+(define_expand "zero_extendqidi2"
+  [(set (match_operand:DI 0 "register_operand" "")
+        (zero_extend:DI (match_operand:QI 1 "register_operand" "")))]
   ""
   "
 {
   if (!TARGET_64BIT)
     {
-      emit_insn (gen_zero_extendqihi2_31 (operands[0], operands[1],
-                              force_const_mem (SImode, const0_rtx)));
+      rtx tmp = gen_reg_rtx (SImode);
+      emit_insn (gen_zero_extendqisi2 (tmp, operands[1]));
+      emit_insn (gen_zero_extendsidi2 (operands[0], tmp));
       DONE;
     }
   else
-   {
-      emit_insn (gen_zero_extendqihi2_64 (operands[0], operands[1]));
+    {
+      operands[1] = gen_lowpart (DImode, operands[1]);
+      emit_insn (gen_ashldi3 (operands[0], operands[1], GEN_INT (56)));
+      emit_insn (gen_lshrdi3 (operands[0], operands[0], GEN_INT (56))); 
       DONE;
     }
-}")
-
+}
+")
 
-;
-; truncdisi2 instruction pattern(s).
-;
-
-(define_insn "truncdisi2"
-  [(set (match_operand:SI 0 "register_operand" "=d")
-        (truncate:SI (match_operand:DI 1 "register_operand" "d")))]
+(define_insn "*zero_extendqidi2"
+  [(set (match_operand:DI 0 "register_operand" "=d")
+        (zero_extend:DI (match_operand:QI 1 "memory_operand" "m")))]
   "TARGET_64BIT"
-  "llgfr\\t%0,%1"
-  [(set_attr "op_type" "RRE")])
-
+  "llgc\\t%0,%1"
+  [(set_attr "op_type" "RXE")
+   (set_attr "atype"   "mem")])
 
 ;
-; truncdihi2 instruction pattern(s).
+; zero_extendhisi2 instruction pattern(s).
 ;
 
-(define_insn "truncdihi2"
-  [(set (match_operand:HI 0 "register_operand" "=d")
-        (truncate:HI (match_operand:DI 1 "register_operand" "d")))]
-  "TARGET_64BIT"
-  "llgfr\\t%0,%1\;iilh\\t%0,0"
-   [(set_attr "op_type" "NN")
-    (set_attr "length" "10")])
-
-
-;
-; truncdiqi2 instruction pattern(s).
-;
+(define_expand "zero_extendhisi2"
+  [(set (match_operand:SI 0 "register_operand" "")
+        (zero_extend:SI (match_operand:HI 1 "register_operand" "")))]
+  ""
+  "
+{
+  operands[1] = gen_lowpart (SImode, operands[1]);
+  emit_insn (gen_andsi3 (operands[0], operands[1], GEN_INT (0xffff)));
+  DONE;
+}
+")
 
-(define_insn "truncdiqi2"
-  [(set (match_operand:QI 0 "register_operand" "=d")
-        (truncate:QI (match_operand:DI 1 "register_operand" "d")))
-   (clobber (reg:CC 33))]
+(define_insn "*zero_extendhisi2_64"
+  [(set (match_operand:SI 0 "register_operand" "=d")
+        (zero_extend:SI (match_operand:HI 1 "memory_operand" "m")))]
   "TARGET_64BIT"
-  "sllg\\t%0,%1,56\;srlg\\t%0,%0,56"
-  [(set_attr "op_type" "NN")
-   (set_attr "type"    "o2")	
-   (set_attr "length"  "12")])
-
-
+  "llgh\\t%0,%1"
+  [(set_attr "op_type" "RXE")
+   (set_attr "atype"   "mem")])
+ 
 ;
-; truncsihi2 instruction pattern(s).
+; zero_extendqisi2 instruction pattern(s).
 ;
 
-(define_expand "truncsihi2"
-  [(set (match_operand:HI 0 "register_operand" "")
-        (truncate:HI (match_operand:SI 1 "register_operand" "")))]
+(define_expand "zero_extendqisi2"
+  [(set (match_operand:SI 0 "register_operand" "")
+        (zero_extend:SI (match_operand:QI 1 "register_operand" "")))]
   ""
   "
 {
-  if (!TARGET_64BIT)
-    {
-      emit_insn (gen_do_truncsihi2 (operands[0], operands[1],
-                              force_const_mem (SImode, const0_rtx)));
-      DONE;
-    }
-}")
-
-
-(define_insn "do_truncsihi2"
-  [(set (match_operand:HI 0 "register_operand" "=d")
-        (truncate:HI (match_operand:SI 1 "register_operand"  "0")))
-        (use (match_operand:SI 2 "memory_operand" "m"))
-   (clobber (reg:CC 33))]
-  ""
-  "icm\\t%0,12,%2"
-   [(set_attr "op_type" "RX")])
+  operands[1] = gen_lowpart (SImode, operands[1]);
+  emit_insn (gen_andsi3 (operands[0], operands[1], GEN_INT (0xff)));
+  DONE;
+}
+")
 
-(define_insn "*truncsihi2_64"
-  [(set (match_operand:HI 0 "register_operand" "=d")
-        (truncate:HI (match_operand:SI 1 "register_operand" "0")))]
+(define_insn "*zero_extendqisi2_64"
+  [(set (match_operand:SI 0 "register_operand" "=d")
+        (zero_extend:SI (match_operand:QI 1 "memory_operand" "m")))]
   "TARGET_64BIT"
-  "iilh\\t%0,0"
-  [(set_attr "op_type" "RI")])
-
-
+  "llgc\\t%0,%1"
+  [(set_attr "op_type" "RXE")
+   (set_attr "atype"   "mem")])
+ 
 ;
-; truncsiqi2 instruction pattern(s).
+; zero_extendqihi2 instruction pattern(s).
 ;
 
-(define_insn "truncsiqi2"
-  [(set (match_operand:QI 0 "register_operand" "=d")
-        (truncate:QI (match_operand:SI 1 "register_operand" "0")))
-   (clobber (reg:CC 33))]
+(define_expand "zero_extendqihi2"
+  [(set (match_operand:HI 0 "register_operand" "")
+        (zero_extend:HI (match_operand:QI 1 "register_operand" "")))]
   "TARGET_64BIT"
-  "iilh\\t%0,0\;nill\\t%0,0x00FF"
-  [(set_attr "op_type" "NN")
-   (set_attr "type"    "o2")
-   (set_attr "length"  "8")])
-
-
-;
-; trunchiqi2 instruction pattern(s).
-;
+  "
+{
+  operands[1] = gen_lowpart (HImode, operands[1]);
+  emit_insn (gen_andhi3 (operands[0], operands[1], GEN_INT (0xff)));
+  DONE;
+}
+")
 
-(define_insn "trunchiqi2"
-  [(set (match_operand:QI 0 "register_operand" "=d")
-        (truncate:QI (match_operand:HI 1 "register_operand" "0")))
+(define_insn "*zero_extendqihi2_64"
+  [(set (match_operand:HI 0 "register_operand" "=d")
+        (zero_extend:HI (match_operand:QI 1 "memory_operand" "m")))
    (clobber (reg:CC 33))]
   "TARGET_64BIT"
-  "nill\\t%0,0x00FF"
-  [(set_attr "op_type" "RI")])
-
+  "llgc\\t%0,%1"
+  [(set_attr "op_type" "RXE")
+   (set_attr "atype"   "mem")])
 
 ;
 ; fixuns_truncdfdi2 and fix_truncdfsi2 instruction pattern(s).
@@ -2338,12 +2609,13 @@
   rtx temp = gen_reg_rtx (DFmode);
   operands[1] = force_reg (DFmode, operands[1]);
 
-  emit_insn (gen_cmpdf (operands[1], force_const_mem (DFmode,
-	CONST_DOUBLE_FROM_REAL_VALUE (0x8000000000000000ULL, DFmode))));
+  emit_insn (gen_cmpdf (operands[1], 
+	CONST_DOUBLE_FROM_REAL_VALUE (
+          REAL_VALUE_ATOF (\"9223372036854775808.0\", DFmode), DFmode)));
   emit_jump_insn (gen_blt (label1));
-
-  emit_insn (gen_subdf3 (temp, operands[1], force_const_mem (DFmode,
-	CONST_DOUBLE_FROM_REAL_VALUE (0x10000000000000000ULL, DFmode))));
+  emit_insn (gen_subdf3 (temp, operands[1],
+	CONST_DOUBLE_FROM_REAL_VALUE (
+          REAL_VALUE_ATOF (\"18446744073709551616.0\", DFmode), DFmode)));
   emit_insn (gen_fix_truncdfdi2_ieee (operands[0], temp, GEN_INT(7)));
   emit_jump (label2);
 
@@ -2372,7 +2644,7 @@
   "TARGET_64BIT && TARGET_HARD_FLOAT && TARGET_IEEE_FLOAT"
   "cgdbr\\t%0,%h2,%1"
   [(set_attr "op_type" "RRE")
-   (set_attr "type"   "other")])
+   (set_attr "type"    "other")])
 
 ;
 ; fixuns_truncdfsi2 and fix_truncdfsi2 instruction pattern(s).
@@ -2389,11 +2661,13 @@
   rtx temp = gen_reg_rtx (DFmode);
 
   operands[1] = force_reg (DFmode,operands[1]);
-  emit_insn (gen_cmpdf (operands[1], force_const_mem (DFmode,
-	CONST_DOUBLE_FROM_REAL_VALUE (0x80000000ULL, DFmode))));
+  emit_insn (gen_cmpdf (operands[1], 
+	CONST_DOUBLE_FROM_REAL_VALUE (
+          REAL_VALUE_ATOF (\"2147483648.0\", DFmode), DFmode)));
   emit_jump_insn (gen_blt (label1));
-  emit_insn (gen_subdf3 (temp, operands[1], force_const_mem (DFmode,
-	CONST_DOUBLE_FROM_REAL_VALUE (0x100000000ULL, DFmode))));
+  emit_insn (gen_subdf3 (temp, operands[1],
+	CONST_DOUBLE_FROM_REAL_VALUE (
+          REAL_VALUE_ATOF (\"4294967296.0\", DFmode), DFmode)));
   emit_insn (gen_fix_truncdfsi2_ieee (operands[0], temp, GEN_INT (7)));
   emit_jump (label2);
 
@@ -2414,12 +2688,8 @@
       /* This is the algorithm from POP chapter A.5.7.2.  */
 
       rtx temp   = assign_stack_local (BLKmode, 2 * UNITS_PER_WORD, BITS_PER_WORD);
-      rtx two31r = force_const_mem (DFmode,
-                                    gen_rtx (CONST_DOUBLE, VOIDmode, cc0_rtx,
-                                             0x08000000, 0x4F000000));
-      rtx two32  = force_const_mem (DFmode,
-                                    gen_rtx (CONST_DOUBLE, VOIDmode, cc0_rtx,
-                                             0x0, 0x4E000001));
+      rtx two31r = s390_gen_rtx_const_DI (0x4f000000, 0x08000000);
+      rtx two32  = s390_gen_rtx_const_DI (0x4e000001, 0x00000000);
 
       operands[1] = force_reg (DFmode, operands[1]);
       emit_insn (gen_fix_truncdfsi2_ibm (operands[0], operands[1], 
@@ -2442,13 +2712,13 @@
   "TARGET_HARD_FLOAT && TARGET_IEEE_FLOAT"
   "cfdbr\\t%0,%h2,%1"
    [(set_attr "op_type" "RRE")
-    (set_attr "type"   "other" )])
+    (set_attr "type"    "other" )])
 
 (define_insn "fix_truncdfsi2_ibm"
   [(set (match_operand:SI 0 "register_operand" "=d")
         (fix:SI (match_operand:DF 1 "nonimmediate_operand" "+f")))
-   (use (match_operand:DF 2 "memory_operand" "m"))
-   (use (match_operand:DF 3 "memory_operand" "m"))
+   (use (match_operand:DI 2 "immediate_operand" "m"))
+   (use (match_operand:DI 3 "immediate_operand" "m"))
    (use (match_operand:BLK 4 "memory_operand" "m"))
    (clobber (reg:CC 33))]
   "TARGET_HARD_FLOAT && TARGET_IBM_FLOAT"
@@ -2461,7 +2731,7 @@
    return \"l\\t%0,%N4\";
 }"
   [(set_attr "op_type" "NN")
-   (set_attr "type"   "other")
+   (set_attr "type"    "other")
    (set_attr "length"  "20")])
 
 ;
@@ -2479,12 +2749,14 @@
   rtx temp = gen_reg_rtx (SFmode);
 
   operands[1] = force_reg (SFmode, operands[1]);
-  emit_insn (gen_cmpsf (operands[1], force_const_mem (SFmode,
-	CONST_DOUBLE_FROM_REAL_VALUE (0x8000000000000000ULL, SFmode))));
+  emit_insn (gen_cmpsf (operands[1], 
+	CONST_DOUBLE_FROM_REAL_VALUE (
+          REAL_VALUE_ATOF (\"9223372036854775808.0\", SFmode), SFmode)));
   emit_jump_insn (gen_blt (label1));
 
-  emit_insn (gen_subsf3 (temp, operands[1], force_const_mem (SFmode,
-	CONST_DOUBLE_FROM_REAL_VALUE (0x10000000000000000ULL, SFmode))));
+  emit_insn (gen_subsf3 (temp, operands[1],
+	CONST_DOUBLE_FROM_REAL_VALUE (
+          REAL_VALUE_ATOF (\"18446744073709551616.0\", SFmode), SFmode)));
   emit_insn (gen_fix_truncsfdi2_ieee (operands[0], temp, GEN_INT(7)));
   emit_jump (label2);
 
@@ -2513,7 +2785,7 @@
   "TARGET_64BIT && TARGET_HARD_FLOAT && TARGET_IEEE_FLOAT"
   "cgebr\\t%0,%h2,%1"
   [(set_attr "op_type" "RRE")
-   (set_attr "type"   "other")])
+   (set_attr "type"    "other")])
 
 ;
 ; fixuns_truncsfsi2 and fix_truncsfsi2 instruction pattern(s).
@@ -2530,11 +2802,13 @@
   rtx temp = gen_reg_rtx (SFmode);
 
   operands[1] = force_reg (SFmode, operands[1]);
-  emit_insn (gen_cmpsf (operands[1], force_const_mem (SFmode,
-	CONST_DOUBLE_FROM_REAL_VALUE (0x80000000ULL, SFmode))));
+  emit_insn (gen_cmpsf (operands[1],
+	CONST_DOUBLE_FROM_REAL_VALUE (
+          REAL_VALUE_ATOF (\"2147483648.0\", SFmode), SFmode)));
   emit_jump_insn (gen_blt (label1));
-  emit_insn (gen_subsf3 (temp, operands[1], force_const_mem (SFmode,
-	CONST_DOUBLE_FROM_REAL_VALUE (0x100000000ULL, SFmode))));
+  emit_insn (gen_subsf3 (temp, operands[1],
+	CONST_DOUBLE_FROM_REAL_VALUE (
+          REAL_VALUE_ATOF (\"4294967296.0\", SFmode), SFmode)));
   emit_insn (gen_fix_truncsfsi2_ieee (operands[0], temp, GEN_INT (7)));
   emit_jump (label2);
 
@@ -2574,7 +2848,7 @@
   "TARGET_HARD_FLOAT && TARGET_IEEE_FLOAT"
   "cfebr\\t%0,%h2,%1"
   [(set_attr "op_type" "RRE")
-   (set_attr "type"   "other")])
+   (set_attr "type"    "other")])
 
 ;
 ; floatdidf2 instruction pattern(s).
@@ -2582,11 +2856,12 @@
 
 (define_insn "floatdidf2"
   [(set (match_operand:DF 0 "register_operand" "=f")
-        (float:DF (match_operand:DI 1 "register_operand" "d")))]
+        (float:DF (match_operand:DI 1 "register_operand" "d")))
+   (clobber (reg:CC 33))]
   "TARGET_64BIT && TARGET_HARD_FLOAT && TARGET_IEEE_FLOAT"
   "cdgbr\\t%0,%1"
   [(set_attr "op_type" "RRE")
-   (set_attr "type"   "other" )])
+   (set_attr "type"    "other" )])
 
 ;
 ; floatdisf2 instruction pattern(s).
@@ -2594,19 +2869,22 @@
 
 (define_insn "floatdisf2"
   [(set (match_operand:SF 0 "register_operand" "=f")
-        (float:SF (match_operand:DI 1 "register_operand" "d")))]
+        (float:SF (match_operand:DI 1 "register_operand" "d")))
+   (clobber (reg:CC 33))]
   "TARGET_64BIT && TARGET_HARD_FLOAT && TARGET_IEEE_FLOAT"
   "cegbr\\t%0,%1"
   [(set_attr "op_type" "RRE")
-   (set_attr "type"   "other" )])
+   (set_attr "type"    "other" )])
 
 ;
 ; floatsidf2 instruction pattern(s).
 ;
 
 (define_expand "floatsidf2"
-  [(set (match_operand:DF 0 "register_operand" "")
-        (float:DF (match_operand:SI 1 "register_operand" "")))]
+  [(parallel
+    [(set (match_operand:DF 0 "register_operand" "")
+          (float:DF (match_operand:SI 1 "register_operand" "")))
+     (clobber (reg:CC 33))])]
   "TARGET_HARD_FLOAT"
   "
 {
@@ -2615,9 +2893,7 @@
       /* This is the algorithm from POP chapter A.5.7.1.  */
 
       rtx temp  = assign_stack_local (BLKmode, 2 * UNITS_PER_WORD, BITS_PER_WORD);
-      rtx two31 = force_const_mem (DFmode,
-                                   gen_rtx (CONST_DOUBLE, VOIDmode, cc0_rtx,
-                                            0x80000000U, 0x4E000000U));
+      rtx two31 = s390_gen_rtx_const_DI (0x4e000000, 0x80000000);	   
 
       emit_insn (gen_floatsidf2_ibm (operands[0], operands[1], two31, temp));
       DONE;
@@ -2626,7 +2902,8 @@
 
 (define_insn "floatsidf2_ieee"
   [(set (match_operand:DF 0 "register_operand" "=f")
-        (float:DF (match_operand:SI 1 "register_operand"  "d")))]
+        (float:DF (match_operand:SI 1 "register_operand"  "d")))
+   (clobber (reg:CC 33))]
   "TARGET_HARD_FLOAT && TARGET_IEEE_FLOAT"
   "cdfbr\\t%0,%1"
   [(set_attr "op_type" "RRE")
@@ -2635,20 +2912,20 @@
 (define_insn "floatsidf2_ibm"
   [(set (match_operand:DF 0 "register_operand" "=f")
         (float:DF (match_operand:SI 1 "register_operand" "d")))
-   (use (match_operand:DF 2 "memory_operand" "m"))
+   (use (match_operand:DI 2 "immediate_operand" "m"))
    (use (match_operand:BLK 3 "memory_operand" "m"))
    (clobber (reg:CC 33))]
   "TARGET_HARD_FLOAT && TARGET_IBM_FLOAT"
   "*
 {
-   output_asm_insn (\"st\\t%0,%N3\", operands);
+   output_asm_insn (\"st\\t%1,%N3\", operands);
    output_asm_insn (\"xi\\t%N3,128\", operands);
    output_asm_insn (\"mvc\\t%O3(4,%R3),%2\", operands);
    output_asm_insn (\"ld\\t%0,%3\", operands);
    return \"sd\\t%0,%2\";
 }"
   [(set_attr "op_type" "NN")
-   (set_attr "type"   "other" )
+   (set_attr "type"    "other" )
    (set_attr "length"  "20")])
 
 ;
@@ -2656,8 +2933,10 @@
 ;
 
 (define_expand "floatsisf2"
-  [(set (match_operand:SF 0 "register_operand" "")
-        (float:SF (match_operand:SI 1 "register_operand" "")))]
+  [(parallel
+    [(set (match_operand:SF 0 "register_operand" "")
+          (float:SF (match_operand:SI 1 "register_operand" "")))
+     (clobber (reg:CC 33))])]
   "TARGET_HARD_FLOAT"
   "
 {
@@ -2673,11 +2952,12 @@
 
 (define_insn "floatsisf2_ieee"
   [(set (match_operand:SF 0 "register_operand" "=f")
-        (float:SF (match_operand:SI 1 "register_operand" "d")))]
+        (float:SF (match_operand:SI 1 "register_operand" "d")))
+   (clobber (reg:CC 33))]
   "TARGET_HARD_FLOAT && TARGET_IEEE_FLOAT"
   "cefbr\\t%0,%1"
   [(set_attr "op_type" "RRE")
-   (set_attr "type"   "other" )])
+   (set_attr "type"    "other" )])
 
 ;
 ; truncdfsf2 instruction pattern(s).
@@ -2687,27 +2967,24 @@
   [(set (match_operand:SF 0 "register_operand" "")
         (float_truncate:SF (match_operand:DF 1 "general_operand" "")))]
   "TARGET_HARD_FLOAT"
-  "
-{
-  if (CONSTANT_P(operands[1]))
-      operands[1] =  force_const_mem (DFmode, operands[1]);
-}")
+  "")
 
 (define_insn "truncdfsf2_ieee"
   [(set (match_operand:SF 0 "register_operand" "=f")
-        (float_truncate:SF (match_operand:DF 1 "nonimmediate_operand" "f")))]
+        (float_truncate:SF (match_operand:DF 1 "general_operand" "f")))]
   "TARGET_HARD_FLOAT && TARGET_IEEE_FLOAT"
   "ledbr\\t%0,%1"
   [(set_attr "op_type"  "RR")])
 
 (define_insn "truncdfsf2_ibm"
   [(set (match_operand:SF 0 "register_operand" "=f,f")
-        (float_truncate:SF (match_operand:DF 1 "nonimmediate_operand" "f,m")))]
+        (float_truncate:SF (match_operand:DF 1 "general_operand" "f,m")))]
   "TARGET_HARD_FLOAT && TARGET_IBM_FLOAT"
   "@
    lrer\\t%0,%1
    le\\t%0,%1"
-  [(set_attr "op_type"  "RR,RX")])
+  [(set_attr "op_type"  "RR,RX")
+   (set_attr "atype"    "reg,mem")])
 
 ;
 ; extendsfdf2 instruction pattern(s).
@@ -2743,7 +3020,9 @@
   "@
    sdr\\t%0,%0\;ler\\t%0,%1
    sdr\\t%0,%0\;le\\t%0,%1"
-  [(set_attr "op_type"  "RRE,RXE")])
+  [(set_attr "op_type"  "RRE,RXE")
+   (set_attr "atype"    "reg,mem")
+   (set_attr "type"     "o2,o2")]) 
 
 
 ;;
@@ -2814,7 +3093,7 @@
   "!TARGET_64BIT"
   "*
 {
-   switch(which_alternative)
+   switch (which_alternative)
      {
      case 0: /* d <- d */
        output_asm_insn (\"ar\\t%0,%2\", operands);
@@ -2825,6 +3104,9 @@
        output_asm_insn (\"a\\t%0,%2\", operands);
        output_asm_insn (\"al\\t%N0,%N2\", operands);
        break;
+
+     default:
+       abort ();
      }
 
    output_asm_insn (\"brc\\t12,.+8\", operands);
@@ -2832,6 +3114,7 @@
 }"
   [(set_attr "op_type" "NN,NN")
    (set_attr "atype"   "reg,mem")
+   (set_attr "type"    "o2,o2")
    (set_attr "length"  "12,16")])
 
 (define_expand "adddi3"
@@ -2883,7 +3166,13 @@
    && !rtx_equal_p (operands[0], operands[2])"
   [(set (match_dup 0) (match_dup 1))
    (set (match_dup 0) (plus:DI (match_dup 0) (match_dup 2)))]
-  "")
+  "
+{
+  if (CONSTANT_P (operands[1]) 
+      && !legitimate_reload_constant_p (operands[1]))
+    operands[1] = force_const_mem (DImode, operands[1]);
+}")
+
 
 (define_split
   [(set (match_operand:DI 0 "register_operand" "")
@@ -2895,23 +3184,36 @@
    && !rtx_equal_p (operands[0], operands[2])"
   [(set (match_dup 0) (match_dup 2))
    (set (match_dup 0) (plus:DI (match_dup 0) (match_dup 1)))]
-  "")
+  "
+{
+  if (CONSTANT_P (operands[2]) 
+      && !legitimate_reload_constant_p (operands[2]))
+    operands[2] = force_const_mem (DImode, operands[2]);
+}")
  
 ;
 ; addsi3 instruction pattern(s).
 ;
-; The following insn is used when it is known that operand one is the stack pointer,
-; and operand two is small enough to fit in the displacement field
-; In this case, the result will be an address
-;
 
-(define_insn "addaddr"
+(define_insn "*la_ccclobber"
+  [(set (match_operand:SI 0 "register_operand" "=a")
+        (match_operand:QI 1 "address_operand"  "p"))
+   (clobber (reg:CC 33))]
+  "legitimate_la_operand_p (operands[1])"
+  "la\\t%0,%a1"
+   [(set_attr "op_type"  "RX")
+    (set_attr "atype"    "mem")
+    (set_attr "type"     "la")])
+
+(define_insn "*addaddr_ccclobber"
   [(set (match_operand:SI 0 "register_operand" "=d,d")
         (plus:SI (match_operand:SI 1 "register_operand" "%a,a")
-                 (match_operand:SI 2 "nonmemory_operand" "J,a")))]
+                 (match_operand:SI 2 "nonmemory_operand" "J,a")))
+   (clobber (reg:CC 33))]
   "(((REGNO (operands[1]) == STACK_POINTER_REGNUM ) ||
+     (REGNO (operands[1]) == FRAME_POINTER_REGNUM ) ||	
      (REGNO (operands[1]) == BASE_REGISTER)) && 
-	(GET_CODE (operands[2]) == REG ||
+        (GET_CODE (operands[2]) == REG ||
 	 CONST_OK_FOR_LETTER_P (INTVAL (operands[2]),'J')))"
   "@
    la\\t%0,%c2(,%1)
@@ -2972,7 +3274,7 @@
   [(set_attr "op_type"  "RR,RI,RX")
    (set_attr "atype"    "reg,reg,mem")])
 
-(define_insn "*do_la"
+(define_insn "*la"
   [(set (match_operand:SI 0 "register_operand" "=a")
         (match_operand:QI 1 "address_operand" "p"))]
   "reload_in_progress || reload_completed
@@ -3008,7 +3310,12 @@
    && !rtx_equal_p (operands[0], operands[2])"
   [(set (match_dup 0) (match_dup 1))
    (set (match_dup 0) (plus:SI (match_dup 0) (match_dup 2)))]
-  "")
+  "
+{
+  if (CONSTANT_P (operands[1]) 
+      && !legitimate_reload_constant_p (operands[1]))
+    operands[1] = force_const_mem (SImode, operands[1]);
+}")
 
 (define_split
   [(set (match_operand:SI 0 "register_operand" "")
@@ -3020,19 +3327,13 @@
    && !rtx_equal_p (operands[0], operands[2])"
   [(set (match_dup 0) (match_dup 2))
    (set (match_dup 0) (plus:SI (match_dup 0) (match_dup 1)))]
-  "")
+  "
+{
+  if (CONSTANT_P (operands[2]) 
+      && !legitimate_reload_constant_p (operands[2]))
+    operands[2] = force_const_mem (SImode, operands[2]);
+}")
  
-(define_insn "addsi_64"
-  [(set (match_operand:SI 0 "register_operand" "=d,d")
-        (plus:SI (match_operand:SI 1 "register_operand" "%a,a")
-                 (match_operand:SI 2 "nonmemory_operand" "J,a")))]
-  "TARGET_64BIT"
-  "@
-   la\\t%0,%c2(,%1)
-   la\\t%0,0(%1,%2)"
-  [(set_attr "op_type"  "RX")
-   (set_attr "atype"    "mem")
-   (set_attr "type"     "la")])
 
 ;
 ; addhi3 instruction pattern(s).
@@ -3077,7 +3378,7 @@
   [(parallel
     [(set (match_operand:DF 0 "register_operand" "=f,f")
           (plus:DF (match_operand:DF 1 "register_operand" "%0,0")
-                   (match_operand:DF 2 "nonimmediate_operand" "f,m")))
+                   (match_operand:DF 2 "general_operand" "f,m")))
      (clobber (reg:CC 33))])]
   "TARGET_HARD_FLOAT"
   "")
@@ -3085,7 +3386,7 @@
 (define_insn "*adddf3"
   [(set (match_operand:DF 0 "register_operand" "=f,f")
         (plus:DF (match_operand:DF 1 "register_operand" "%0,0")
-                 (match_operand:DF 2 "nonimmediate_operand" "f,m")))
+                 (match_operand:DF 2 "general_operand" "f,m")))
    (clobber (reg:CC 33))]
   "TARGET_HARD_FLOAT && TARGET_IEEE_FLOAT"
   "@
@@ -3097,7 +3398,7 @@
 (define_insn "*adddf3_ibm"
   [(set (match_operand:DF 0 "register_operand" "=f,f")
         (plus:DF (match_operand:DF 1 "register_operand" "%0,0")
-                 (match_operand:DF 2 "nonimmediate_operand" "f,m")))
+                 (match_operand:DF 2 "general_operand" "f,m")))
    (clobber (reg:CC 33))]
   "TARGET_HARD_FLOAT && TARGET_IBM_FLOAT"
   "@
@@ -3114,7 +3415,7 @@
   [(parallel
     [(set (match_operand:SF 0 "register_operand" "=f,f")
           (plus:SF (match_operand:SF 1 "register_operand" "%0,0")
-                   (match_operand:SF 2 "nonimmediate_operand" "f,m")))
+                   (match_operand:SF 2 "general_operand" "f,m")))
      (clobber (reg:CC 33))])]
   "TARGET_HARD_FLOAT"
   "")
@@ -3122,7 +3423,7 @@
 (define_insn "*addsf3"
   [(set (match_operand:SF 0 "register_operand" "=f,f")
         (plus:SF (match_operand:SF 1 "register_operand" "%0,0")
-                 (match_operand:SF 2 "nonimmediate_operand" "f,m")))
+                 (match_operand:SF 2 "general_operand" "f,m")))
    (clobber (reg:CC 33))]
   "TARGET_HARD_FLOAT && TARGET_IEEE_FLOAT"
   "@
@@ -3134,7 +3435,7 @@
 (define_insn "*addsf3"
   [(set (match_operand:SF 0 "register_operand" "=f,f")
         (plus:SF (match_operand:SF 1 "register_operand" "%0,0")
-                 (match_operand:SF 2 "nonimmediate_operand" "f,m")))
+                 (match_operand:SF 2 "general_operand" "f,m")))
    (clobber (reg:CC 33))]
   "TARGET_HARD_FLOAT && TARGET_IBM_FLOAT"
   "@
@@ -3167,7 +3468,7 @@
 (define_insn "subdi3"
   [(set (match_operand:DI 0 "register_operand" "=d,d")
         (minus:DI (match_operand:DI 1 "register_operand" "0,0")
-                  (match_operand:DI 2 "nonimmediate_operand" "d,m")))
+                  (match_operand:DI 2 "general_operand" "d,m")))
    (clobber (reg:CC 33))]
   ""
   "*
@@ -3182,6 +3483,9 @@
        output_asm_insn (\"s\\t%0,%2\", operands);
        output_asm_insn (\"sl\\t%N0,%N2\", operands);
        break;
+
+     default:
+       abort ();
      }
 
    output_asm_insn (\"brc\\t11,.+8\", operands);
@@ -3189,6 +3493,7 @@
 }"
   [(set_attr "op_type"  "NN,NN")
    (set_attr "atype"    "reg,mem")
+   (set_attr "type"     "other,other")
    (set_attr "length"   "12,16")])
 
 ;
@@ -3239,17 +3544,16 @@
 ;
 
 (define_insn "subhi3"
-  [(set (match_operand:HI 0 "register_operand" "=d,d,d")
-        (minus:HI (match_operand:HI 1 "register_operand" "0,0,0")
-                  (match_operand:HI 2 "nonimmediate_operand" "d,K,m")))
+  [(set (match_operand:HI 0 "register_operand" "=d,d")
+        (minus:HI (match_operand:HI 1 "register_operand" "0,0")
+                  (match_operand:HI 2 "general_operand" "d,m")))
    (clobber (reg:CC 33))]
   ""
   "@
    sr\\t%0,%2
-   ahi\\t%0,-%h2
    sh\\t%0,%2"
-  [(set_attr "op_type"  "RR,RI,RX")
-   (set_attr "atype"    "reg,reg,mem")])
+  [(set_attr "op_type"  "RR,RX")
+   (set_attr "atype"    "reg,mem")])
 
 ;
 ; subqi3 instruction pattern(s).
@@ -3272,7 +3576,7 @@
   [(parallel
     [(set (match_operand:DF 0 "register_operand" "=f,f")
           (minus:DF (match_operand:DF 1 "register_operand" "0,0")
-                    (match_operand:DF 2 "nonimmediate_operand" "f,m")))
+                    (match_operand:DF 2 "general_operand" "f,m")))
      (clobber (reg:CC 33))])]
   "TARGET_HARD_FLOAT"
   "")
@@ -3280,7 +3584,7 @@
 (define_insn "*subdf3"
   [(set (match_operand:DF 0 "register_operand" "=f,f")
         (minus:DF (match_operand:DF 1 "register_operand" "0,0")
-                  (match_operand:DF 2 "nonimmediate_operand" "f,m")))
+                  (match_operand:DF 2 "general_operand" "f,m")))
    (clobber (reg:CC 33))]
   "TARGET_HARD_FLOAT && TARGET_IEEE_FLOAT"
   "@
@@ -3292,7 +3596,7 @@
 (define_insn "*subdf3_ibm"
   [(set (match_operand:DF 0 "register_operand" "=f,f")
         (minus:DF (match_operand:DF 1 "register_operand" "0,0")
-                  (match_operand:DF 2 "nonimmediate_operand" "f,m")))
+                  (match_operand:DF 2 "general_operand" "f,m")))
    (clobber (reg:CC 33))]
   "TARGET_HARD_FLOAT && TARGET_IBM_FLOAT"
   "@
@@ -3309,7 +3613,7 @@
   [(parallel
     [(set (match_operand:SF 0 "register_operand" "=f,f")
           (minus:SF (match_operand:SF 1 "register_operand" "0,0")
-                    (match_operand:SF 2 "nonimmediate_operand" "f,m")))
+                    (match_operand:SF 2 "general_operand" "f,m")))
      (clobber (reg:CC 33))])]
   "TARGET_HARD_FLOAT"
   "")
@@ -3317,7 +3621,7 @@
 (define_insn "*subsf3"
   [(set (match_operand:SF 0 "register_operand" "=f,f")
         (minus:SF (match_operand:SF 1 "register_operand" "0,0")
-                  (match_operand:SF 2 "nonimmediate_operand" "f,m")))
+                  (match_operand:SF 2 "general_operand" "f,m")))
    (clobber (reg:CC 33))]
   "TARGET_HARD_FLOAT && TARGET_IEEE_FLOAT"
   "@
@@ -3329,7 +3633,7 @@
 (define_insn "*subsf3_ibm"
   [(set (match_operand:SF 0 "register_operand" "=f,f")
         (minus:SF (match_operand:SF 1 "register_operand" "0,0")
-                  (match_operand:SF 2 "nonimmediate_operand" "f,m")))
+                  (match_operand:SF 2 "general_operand" "f,m")))
    (clobber (reg:CC 33))]
   "TARGET_HARD_FLOAT && TARGET_IBM_FLOAT"
   "@
@@ -3343,53 +3647,11 @@
 ;;- Multiply instructions.
 ;;
 
-(define_expand "muldi3"
-  [(parallel
-    [(set (match_operand:DI 0 "register_operand" "")
-          (mult:DI (match_operand:DI 1 "register_operand" "")
-                   (match_operand:DI 2 "register_operand" "")))
-     (clobber (reg:CC 33))])]
-  ""
-  "
-{
-  if (!TARGET_64BIT)
-    {
-      rtx label1 = gen_label_rtx ();
-      rtx label2 = gen_label_rtx ();
-      rtx op0_0 = operand_subword (operands[0], 0 ,1, DImode);
-      rtx op0_1 = operand_subword (operands[0], 1 ,1, DImode);
-      rtx temp1_0 = gen_reg_rtx (SImode); 
-      rtx temp1_1 = gen_reg_rtx (SImode);
-      rtx temp2_0 = gen_reg_rtx (SImode);
-      rtx temp2_1 = gen_reg_rtx (SImode);
- 
-      emit_move_insn (temp1_0, operand_subword (operands[1], 0 ,1, DImode)); 
-      emit_move_insn (temp1_1, operand_subword (operands[1], 1 ,1, DImode));
-      emit_move_insn (temp2_0, operand_subword (operands[2], 0 ,1, DImode));
-      emit_move_insn (temp2_1, operand_subword (operands[2], 1 ,1, DImode));
-      emit_move_insn (op0_1, temp1_1);
-      emit_insn (gen_mulsi_6432 (operands[0], operands[0], temp2_1));
-      
-      emit_insn (gen_cmpsi (temp1_1, const0_rtx));
-      emit_jump_insn (gen_bge (label1));
-      emit_insn (gen_addsi3 (op0_0, op0_0, temp2_1));
-      emit_label (label1);
-      emit_insn (gen_cmpsi (temp2_1, const0_rtx));
-      emit_jump_insn (gen_bge (label2));
-      emit_insn (gen_addsi3 (op0_0, op0_0, temp1_1));
-      emit_label (label2);
-
-      emit_insn (gen_mulsi3 (temp2_1, temp2_1, temp1_0));
-      emit_insn (gen_addsi3 (op0_0, op0_0, temp2_1));
-      
-      emit_insn (gen_mulsi3 (temp1_1, temp1_1, temp2_0));
-      emit_insn (gen_addsi3 (op0_0, op0_0, temp1_1));
-      
-      DONE;
-    }
-}")
+;
+; muldi3 instruction pattern(s).
+;
 
-(define_insn "*muldi3_64"
+(define_insn "muldi3"
   [(set (match_operand:DI 0 "register_operand" "=d,d,d")
         (mult:DI (match_operand:DI 1 "register_operand" "%0,0,0")
                  (match_operand:DI 2 "general_operand" "d,K,m")))
@@ -3404,35 +3666,6 @@
    (set_attr "type"     "imul")])
 
 ;
-; mulsidi3 instruction pattern(s).
-;
-
-;(define_expand "mulsidi3"
-;  [(set (match_operand:DI 0 "register_operand" "")
-;	(mult:DI (sign_extend:DI (match_operand:SI 1 "register_operand" ""))
-;		 (sign_extend:DI (match_operand:SI 2 "general_operand" ""))))]
-;  ""
-;  "
-;{
-;  emit_insn (gen_extendsidi2 (operands[0], operands[1]));
-;  emit_insn (gen_muldisidi3  (operands[0], operands[0], operands[2]));
-;  DONE;
-;}")
-
-;(define_insn "muldisidi3"
-;  [(set (match_operand:DI 0 "register_operand" "=d,d")
-;	(mult:DI (match_operand:DI 1 "register_operand" "0,0")
-;		 (sign_extend:DI (match_operand:SI 2 "general_operand" "d,m"))))
-;    (clobber (reg:CC 33))]
-;   "!TARGET_64BIT"
-;   "@
-;    mr\\t%0,%2
-;    m\\t%0,%2"
-;  [(set_attr "op_type"  "RR,RX")
-;   (set_attr "atype"    "reg,mem")
-;   (set_attr "type"     "imul")])
-
-;
 ; mulsi3 instruction pattern(s).
 ;
 
@@ -3450,12 +3683,38 @@
    (set_attr "atype"    "reg,reg,mem")
    (set_attr "type"     "imul")])
 
+;
+; mulsidi3 instruction pattern(s).
+;
+
+(define_expand "mulsidi3"
+  [(set (match_operand:DI 0 "register_operand" "")
+	(mult:DI (sign_extend:DI (match_operand:SI 1 "register_operand" ""))
+		 (sign_extend:DI (match_operand:SI 2 "nonimmediate_operand" ""))))]
+  "!TARGET_64BIT"
+  "
+{
+  rtx insn;
+
+  emit_insn (gen_rtx_CLOBBER (VOIDmode, operands[0]));
+  emit_move_insn (gen_lowpart (SImode, operands[0]), operands[1]);
+  insn = emit_insn (gen_mulsi_6432 (operands[0], operands[0], operands[2]));
+
+  REG_NOTES (insn) =
+	gen_rtx_EXPR_LIST (REG_EQUAL, 
+                           gen_rtx_MULT (DImode, 
+				gen_rtx_SIGN_EXTEND (DImode, operands[1]),
+				gen_rtx_SIGN_EXTEND (DImode, operands[2])),
+			   REG_NOTES (insn));
+  DONE;
+}")
+ 
 (define_insn "mulsi_6432"
    [(set (match_operand:DI 0 "register_operand" "=d,d")
          (mult:DI (sign_extend:DI 
-	            (subreg:SI (match_operand:DI 1 "register_operand" "0,0") 4))
+	            (truncate:SI (match_operand:DI 1 "register_operand" "0,0")))
                   (sign_extend:DI
-	            (match_operand:SI 2 "general_operand" "d,m"))))
+	            (match_operand:SI 2 "nonimmediate_operand" "d,m"))))
     (clobber (reg:CC 33))]
    "!TARGET_64BIT"
    "@
@@ -3465,7 +3724,6 @@
    (set_attr "atype"    "reg,mem")
    (set_attr "type"     "imul")])
  
-
 ;
 ; muldf3 instruction pattern(s).
 ;
@@ -3474,7 +3732,7 @@
   [(parallel
     [(set (match_operand:DF 0 "register_operand" "=f,f")
           (mult:DF (match_operand:DF 1 "register_operand" "%0,0")
-                   (match_operand:DF 2 "nonimmediate_operand" "f,m")))
+                   (match_operand:DF 2 "general_operand" "f,m")))
      (clobber (reg:CC 33))])]
   "TARGET_HARD_FLOAT"
   "")
@@ -3482,20 +3740,20 @@
 (define_insn "*muldf3"
   [(set (match_operand:DF 0 "register_operand" "=f,f")
         (mult:DF (match_operand:DF 1 "register_operand" "%0,0")
-                 (match_operand:DF 2 "nonimmediate_operand" "f,m")))
+                 (match_operand:DF 2 "general_operand" "f,m")))
    (clobber (reg:CC 33))]
   "TARGET_HARD_FLOAT && TARGET_IEEE_FLOAT"
   "@
    mdbr\\t%0,%2
    mdb\\t%0,%2"
   [(set_attr "op_type"  "RR,RX")
-   (set_attr "type"    "fmul")
+   (set_attr "type"     "fmul")
    (set_attr "atype"    "reg,mem")])
 
 (define_insn "*muldf3_ibm"
   [(set (match_operand:DF 0 "register_operand" "=f,f")
         (mult:DF (match_operand:DF 1 "register_operand" "%0,0")
-                 (match_operand:DF 2 "nonimmediate_operand" "f,m")))
+                 (match_operand:DF 2 "general_operand" "f,m")))
    (clobber (reg:CC 33))]
   "TARGET_HARD_FLOAT && TARGET_IBM_FLOAT"
   "@
@@ -3513,7 +3771,7 @@
   [(parallel
     [(set (match_operand:SF 0 "register_operand" "=f,f")
           (mult:SF (match_operand:SF 1 "register_operand" "%0,0")
-                   (match_operand:SF 2 "nonimmediate_operand" "f,m")))
+                   (match_operand:SF 2 "general_operand" "f,m")))
      (clobber (reg:CC 33))])]
   "TARGET_HARD_FLOAT"
   "")
@@ -3521,7 +3779,7 @@
 (define_insn "*mulsf3"
   [(set (match_operand:SF 0 "register_operand" "=f,f")
         (mult:SF (match_operand:SF 1 "register_operand" "%0,0")
-                 (match_operand:SF 2 "nonimmediate_operand" "f,m")))
+                 (match_operand:SF 2 "general_operand" "f,m")))
    (clobber (reg:CC 33))]
   "TARGET_HARD_FLOAT && TARGET_IEEE_FLOAT"
   "@
@@ -3534,7 +3792,7 @@
 (define_insn "*mulsf3_ibm"
   [(set (match_operand:SF 0 "register_operand" "=f,f")
         (mult:SF (match_operand:SF 1 "register_operand" "%0,0")
-                 (match_operand:SF 2 "nonimmediate_operand" "f,m")))
+                 (match_operand:SF 2 "general_operand" "f,m")))
    (clobber (reg:CC 33))]
   "TARGET_HARD_FLOAT && TARGET_IBM_FLOAT"
   "@
@@ -3550,56 +3808,58 @@
 ;;
 
 ;
-; divdi3 and moddi3 instruction pattern(s).
+; divmoddi4 instruction pattern(s).
 ;
 
-(define_expand "divdi3"
-  [(set (match_operand:DI 0 "register_operand" "=d")
-        (div:DI (match_operand:DI 1 "register_operand" "d")
-                (match_operand:DI 2 "general_operand" "")))]
-  "TARGET_64BIT"
-  "
-{
-  rtx op3 = gen_reg_rtx (TImode);
-
-  if (CONSTANT_P (operands[2]))
-    operands[2] = force_const_mem (DImode, operands[2]);
-
-  emit_move_insn (gen_rtx_SUBREG (DImode, op3, 0), const0_rtx);	
-  emit_move_insn (gen_rtx_SUBREG (DImode, op3, 8), operands[1]);	
-  emit_insn (gen_divmodtidi3 (op3, op3, operands[2]));
-  emit_move_insn (operands[0], gen_rtx_SUBREG (DImode, op3, 8));
-  DONE;
-}")
-
-(define_expand "moddi3"
-  [(set (match_operand:DI 0 "register_operand" "=d")
-        (mod:DI (match_operand:DI 1 "register_operand" "d")
-                (match_operand:DI 2 "general_operand" "")))]
+(define_expand "divmoddi4"
+  [(parallel [(set (match_operand:DI 0 "general_operand" "")
+		   (div:DI (match_operand:DI 1 "general_operand" "")
+			   (match_operand:DI 2 "general_operand" "")))
+	      (set (match_operand:DI 3 "general_operand" "")
+		   (mod:DI (match_dup 1) (match_dup 2)))])
+   (clobber (match_dup 4))]
   "TARGET_64BIT"
   "
 {
-  rtx op3 = gen_reg_rtx (TImode);
-
-  if (CONSTANT_P (operands[2]))
-    operands[2] = force_const_mem (DImode, operands[2]);	
+  rtx insn, div_equal, mod_equal, equal;
+
+  div_equal = gen_rtx_DIV (DImode, operands[1], operands[2]);
+  mod_equal = gen_rtx_MOD (DImode, operands[1], operands[2]);
+  equal = gen_rtx_IOR (TImode,
+		       gen_rtx_ZERO_EXTEND (TImode, div_equal),
+		       gen_rtx_ASHIFT (TImode,
+				       gen_rtx_ZERO_EXTEND (TImode, mod_equal),
+				       GEN_INT (64)));
+
+  operands[4] = gen_reg_rtx(TImode);
+  emit_insn (gen_rtx_CLOBBER (VOIDmode, operands[4]));
+  emit_move_insn (gen_lowpart (DImode, operands[4]), operands[1]);
+  insn = emit_insn (gen_divmodtidi3 (operands[4], operands[4], operands[2]));
+  REG_NOTES (insn) =
+	gen_rtx_EXPR_LIST (REG_EQUAL, equal, REG_NOTES (insn));
+
+  insn = emit_move_insn (operands[0], gen_lowpart (DImode, operands[4]));
+  REG_NOTES (insn) =
+        gen_rtx_EXPR_LIST (REG_EQUAL, div_equal, REG_NOTES (insn));
+
+  insn = emit_move_insn (operands[3], gen_highpart (DImode, operands[4]));
+  REG_NOTES (insn) =
+        gen_rtx_EXPR_LIST (REG_EQUAL, mod_equal, REG_NOTES (insn));
 
-  emit_move_insn (gen_rtx_SUBREG (DImode, op3, 0), const0_rtx);	
-  emit_move_insn (gen_rtx_SUBREG (DImode, op3, 8), operands[1]);	
-  emit_insn (gen_divmodtidi3 (op3, op3, operands[2]));
-  emit_move_insn (operands[0], gen_rtx_SUBREG (DImode, op3, 0));
   DONE;
 }")
 
 (define_insn "divmodtidi3"
-  [(set (subreg:DI (match_operand:TI 0 "register_operand" "=d,d") 0)
-        (truncate:DI 
-          (mod:TI (match_operand:TI 1 "register_operand" "0,0")
-                  (sign_extend:TI 
-                    (match_operand:DI 2 "nonimmediate_operand" "d,m")))))
-   (set (subreg:DI (match_dup 0) 8)
-        (truncate:DI (div:TI (match_dup 1) (sign_extend:TI (match_dup 2)))))
-   (clobber (reg:CC 33))]
+  [(set (match_operand:TI 0 "register_operand" "=d,d")
+        (ior:TI
+          (zero_extend:TI
+            (div:DI (truncate:DI (match_operand:TI 1 "register_operand" "0,0"))
+                    (match_operand:DI 2 "general_operand" "d,m")))
+          (ashift:TI
+            (zero_extend:TI
+              (mod:DI (truncate:DI (match_dup 1)) 
+                      (match_dup 2)))
+            (const_int 64))))]
   "TARGET_64BIT"
   "@
    dsgr\\t%0,%2
@@ -3608,57 +3868,80 @@
    (set_attr "type"     "idiv")
    (set_attr "atype"    "reg,mem")])
 
-;
-; udivdi3 and umoddi3 instruction pattern(s).
-;
-
-(define_expand "udivdi3"
-  [(set (match_operand:DI 0 "register_operand" "=d")
-        (udiv:DI (match_operand:DI 1 "register_operand" "d")
-                (match_operand:DI 2 "general_operand" "")))]
+(define_insn "divmodtisi3"
+  [(set (match_operand:TI 0 "register_operand" "=d,d")
+        (ior:TI
+          (zero_extend:TI
+            (div:DI (truncate:DI (match_operand:TI 1 "register_operand" "0,0"))
+                    (sign_extend:DI (match_operand:SI 2 "nonimmediate_operand" "d,m"))))
+          (ashift:TI
+            (zero_extend:TI
+              (mod:DI (truncate:DI (match_dup 1)) 
+                      (sign_extend:DI (match_dup 2))))
+            (const_int 64))))]
   "TARGET_64BIT"
-  "
-{
-  rtx op3 = gen_reg_rtx(TImode);
-
-  if (CONSTANT_P (operands[2]))
-    operands[2] = force_const_mem (DImode, operands[2]);
+  "@
+   dsgfr\\t%0,%2
+   dsgf\\t%0,%2"
+  [(set_attr "op_type"  "RRE,RXE")
+   (set_attr "type"     "idiv")
+   (set_attr "atype"    "reg,mem")])
 
-  emit_move_insn (gen_rtx_SUBREG (DImode, op3, 0), const0_rtx);	
-  emit_move_insn (gen_rtx_SUBREG (DImode, op3, 8), operands[1]);
-  emit_insn (gen_udivmodtidi3 (op3, op3, operands[2]));
-  emit_move_insn (operands[0], gen_rtx_SUBREG (DImode, op3, 8));
-  DONE;
-}")
+;
+; udivmoddi4 instruction pattern(s).
+;
 
-(define_expand "umoddi3"
-  [(set (match_operand:DI 0 "register_operand" "=d")
-        (umod:DI (match_operand:DI 1 "register_operand" "d")
-                (match_operand:DI 2 "general_operand" "")))]
+(define_expand "udivmoddi4"
+  [(parallel [(set (match_operand:DI 0 "general_operand" "")
+		   (udiv:DI (match_operand:DI 1 "general_operand" "")
+			    (match_operand:DI 2 "nonimmediate_operand" "")))
+	      (set (match_operand:DI 3 "general_operand" "")
+		   (umod:DI (match_dup 1) (match_dup 2)))])
+   (clobber (match_dup 4))]
   "TARGET_64BIT"
   "
 {
-  rtx op3 = gen_reg_rtx (TImode);
+  rtx insn, div_equal, mod_equal, equal;
+
+  div_equal = gen_rtx_UDIV (DImode, operands[1], operands[2]);
+  mod_equal = gen_rtx_UMOD (DImode, operands[1], operands[2]);
+  equal = gen_rtx_IOR (TImode,
+		       gen_rtx_ZERO_EXTEND (TImode, div_equal),
+		       gen_rtx_ASHIFT (TImode,
+				       gen_rtx_ZERO_EXTEND (TImode, mod_equal),
+				       GEN_INT (64)));
+
+  operands[4] = gen_reg_rtx(TImode);
+  emit_insn (gen_rtx_CLOBBER (VOIDmode, operands[4]));
+  emit_move_insn (gen_lowpart (DImode, operands[4]), operands[1]);
+  emit_move_insn (gen_highpart (DImode, operands[4]), const0_rtx);
+  insn = emit_insn (gen_udivmodtidi3 (operands[4], operands[4], operands[2]));
+  REG_NOTES (insn) =
+	gen_rtx_EXPR_LIST (REG_EQUAL, equal, REG_NOTES (insn));
+
+  insn = emit_move_insn (operands[0], gen_lowpart (DImode, operands[4]));
+  REG_NOTES (insn) =
+        gen_rtx_EXPR_LIST (REG_EQUAL, div_equal, REG_NOTES (insn));
+
+  insn = emit_move_insn (operands[3], gen_highpart (DImode, operands[4]));
+  REG_NOTES (insn) =
+        gen_rtx_EXPR_LIST (REG_EQUAL, mod_equal, REG_NOTES (insn));
 
-  if (CONSTANT_P (operands[2]))
-    operands[2] = force_const_mem (DImode, operands[2]);
-
-  emit_move_insn (gen_rtx_SUBREG (DImode, op3, 0), const0_rtx);	
-  emit_move_insn (gen_rtx_SUBREG (DImode, op3, 8), operands[1]);	
-  emit_insn (gen_udivmodtidi3 (op3, op3, operands[2]));
-  emit_move_insn (operands[0], gen_rtx_SUBREG (DImode, op3, 0));
   DONE;
 }")
 
 (define_insn "udivmodtidi3"
-  [(set (subreg:DI (match_operand:TI 0 "register_operand" "=d,d") 0)
-        (truncate:DI 
-          (umod:TI (match_operand:TI 1 "register_operand" "0,0")
-                   (zero_extend:TI 
-                     (match_operand:DI 2 "nonimmediate_operand" "d,m")))))
-   (set (subreg:DI (match_dup 0) 8)
-        (truncate:DI (udiv:TI (match_dup 1) (zero_extend:TI (match_dup 2)))))
-   (clobber (reg:CC 33))]
+  [(set (match_operand:TI 0 "register_operand" "=d,d")
+        (ior:TI (zero_extend:TI 
+                  (truncate:DI
+                    (udiv:TI (match_operand:TI 1 "register_operand" "0,0")
+                             (zero_extend:TI
+                               (match_operand:DI 2 "nonimmediate_operand" "d,m")))))
+                (ashift:TI
+                  (zero_extend:TI
+                    (truncate:DI
+                      (umod:TI (match_dup 1) (zero_extend:TI (match_dup 2)))))
+                  (const_int 64))))]
   "TARGET_64BIT"
   "@
    dlgr\\t%0,%2
@@ -3668,62 +3951,58 @@
    (set_attr "atype"    "reg,mem")])
 
 ;
-; divsi3 and modsi3 instruction pattern(s).
+; divmodsi4 instruction pattern(s).
 ;
 
-(define_expand "divsi3"
-  [(set (match_operand:SI 0 "register_operand" "=d")
-        (div:SI (match_operand:SI 1 "register_operand" "d")
-                (match_operand:SI 2 "nonimmediate_operand" "")))]
+(define_expand "divmodsi4"
+  [(parallel [(set (match_operand:SI 0 "general_operand" "")
+		   (div:SI (match_operand:SI 1 "general_operand" "")
+			   (match_operand:SI 2 "nonimmediate_operand" "")))
+	      (set (match_operand:SI 3 "general_operand" "")
+		   (mod:SI (match_dup 1) (match_dup 2)))])
+   (clobber (match_dup 4))]
   "!TARGET_64BIT"
   "
 {
-  rtx tmp = gen_reg_rtx (DImode);
+  rtx insn, div_equal, mod_equal, equal;
+
+  div_equal = gen_rtx_DIV (SImode, operands[1], operands[2]);
+  mod_equal = gen_rtx_MOD (SImode, operands[1], operands[2]);
+  equal = gen_rtx_IOR (DImode,
+		       gen_rtx_ZERO_EXTEND (DImode, div_equal),
+		       gen_rtx_ASHIFT (DImode,
+				       gen_rtx_ZERO_EXTEND (DImode, mod_equal),
+				       GEN_INT (32)));
+
+  operands[4] = gen_reg_rtx(DImode);
+  emit_insn (gen_extendsidi2 (operands[4], operands[1]));
+  insn = emit_insn (gen_divmoddisi3 (operands[4], operands[4], operands[2]));
+  REG_NOTES (insn) =
+	gen_rtx_EXPR_LIST (REG_EQUAL, equal, REG_NOTES (insn));
+
+  insn = emit_move_insn (operands[0], gen_lowpart (SImode, operands[4]));
+  REG_NOTES (insn) =
+        gen_rtx_EXPR_LIST (REG_EQUAL, div_equal, REG_NOTES (insn));
+
+  insn = emit_move_insn (operands[3], gen_highpart (SImode, operands[4]));
+  REG_NOTES (insn) =
+        gen_rtx_EXPR_LIST (REG_EQUAL, mod_equal, REG_NOTES (insn));
 
-  if (CONSTANT_P (operands[2]))
-    operands[2] = force_const_mem (SImode, operands[2]);	
-  else
-    operands[2] = force_reg (SImode, operands[2]);
-
-  emit_insn (gen_rtx_CLOBBER (SImode, gen_rtx_SUBREG (SImode, tmp, 4)));
-  emit_move_insn (gen_rtx_SUBREG (SImode, tmp, 0), operands[1]);
-  emit_insn (gen_ashrdi3 (tmp, tmp, GEN_INT (32))); 
-  emit_insn (gen_divmoddisi3 (tmp, tmp, operands[2]));
-  emit_move_insn (operands[0], gen_rtx_SUBREG (SImode, tmp, 4));
-  DONE;
-}")
-
-(define_expand "modsi3"
-  [(set (match_operand:SI 0 "register_operand" "=d")
-        (mod:SI (match_operand:SI 1 "register_operand" "d")
-                (match_operand:SI 2 "nonimmediate_operand" "")))]
-  "!TARGET_64BIT"
-  "
-{
-  rtx tmp = gen_reg_rtx (DImode);
-
-  if (CONSTANT_P (operands[2]))
-    operands[2] = force_const_mem (SImode, operands[2]);	
-  else
-    operands[2] = force_reg (SImode, operands[2]);
-
-  emit_insn (gen_rtx_CLOBBER (SImode, gen_rtx_SUBREG (SImode, tmp, 4)));
-  emit_insn (gen_movsi (gen_rtx_SUBREG (SImode, tmp, 0), operands[1]));
-  emit_insn (gen_ashrdi3 (tmp, tmp, GEN_INT (32))); 
-  emit_insn (gen_divmoddisi3 (tmp, tmp, operands[2]));
-  emit_move_insn (operands[0], gen_rtx_SUBREG (SImode, tmp, 0));
   DONE;
 }")
 
 (define_insn "divmoddisi3"
-  [(set (subreg:SI (match_operand:DI 0 "register_operand" "=d,d") 0)
-        (truncate:SI 
-          (mod:DI (match_operand:DI 1 "register_operand" "0,0")
-                  (sign_extend:DI 
-                    (match_operand:SI 2 "nonimmediate_operand" "d,m")))))
-   (set (subreg:SI (match_dup 0) 4)
-        (truncate:SI (div:DI (match_dup 1) (sign_extend:DI (match_dup 2)))))
-   (clobber (reg:CC 33))]
+  [(set (match_operand:DI 0 "register_operand" "=d,d")
+        (ior:DI (zero_extend:DI
+                  (truncate:SI
+                    (div:DI (match_operand:DI 1 "register_operand" "0,0")
+                            (sign_extend:DI 
+                              (match_operand:SI 2 "nonimmediate_operand" "d,m")))))
+                (ashift:DI
+                  (zero_extend:DI
+                    (truncate:SI
+                      (mod:DI (match_dup 1) (sign_extend:SI (match_dup 2)))))
+                  (const_int 32))))]
   "!TARGET_64BIT"
   "@
    dr\\t%0,%2
@@ -3736,18 +4015,26 @@
 ; udivsi3 and umodsi3 instruction pattern(s).
 ;
 
+
 (define_expand "udivsi3"
   [(set (match_operand:SI 0 "register_operand" "=d")
         (udiv:SI (match_operand:SI 1 "general_operand" "")
-                 (match_operand:SI 2 "general_operand" "")))]
+                 (match_operand:SI 2 "general_operand" "")))
+   (clobber (match_dup 3))]
   "!TARGET_64BIT"
   "
 {
-  rtx dr_0, dr_1, tmp;
+  rtx insn, udiv_equal, umod_equal, equal;
+
+  udiv_equal = gen_rtx_UDIV (SImode, operands[1], operands[2]);
+  umod_equal = gen_rtx_UMOD (SImode, operands[1], operands[2]);
+  equal = gen_rtx_IOR (DImode,
+		       gen_rtx_ZERO_EXTEND (DImode, udiv_equal),
+		       gen_rtx_ASHIFT (DImode,
+				       gen_rtx_ZERO_EXTEND (DImode, umod_equal),
+				       GEN_INT (32)));
 
-  tmp = gen_reg_rtx (DImode);
-  dr_0 = gen_rtx_SUBREG (SImode, tmp, 0);	
-  dr_1 = gen_rtx_SUBREG (SImode, tmp, 4);
+  operands[3] = gen_reg_rtx (DImode);
 
   if (CONSTANT_P (operands[2]))
     {
@@ -3755,19 +4042,29 @@
         {
           rtx label1 = gen_label_rtx ();
 
-          emit_move_insn (dr_0, operands[1]);
-          emit_move_insn (dr_1, const0_rtx);
-          emit_insn (gen_cmpsi (dr_0, operands[2]));
+	  operands[1] = make_safe_from (operands[1], operands[0]);
+          emit_move_insn (operands[0], const0_rtx);
+          emit_insn (gen_cmpsi (operands[1], operands[2]));
           emit_jump_insn (gen_bltu (label1));
-          emit_move_insn (dr_1, const1_rtx);
+          emit_move_insn (operands[0], const1_rtx);
           emit_label (label1);
         }
       else
         {
-          operands[2] = force_const_mem (SImode, operands[2]);
-          emit_move_insn (gen_rtx_SUBREG (SImode, tmp, 0), const0_rtx);
-          emit_move_insn (gen_rtx_SUBREG (SImode, tmp, 4), operands[1]);
-  	  emit_insn (gen_divmoddisi3 (tmp, tmp, operands[2]));
+          operands[2] = force_reg (SImode, operands[2]);	
+          operands[2] = make_safe_from (operands[2], operands[0]);	
+
+	  emit_insn (gen_zero_extendsidi2 (operands[3], operands[1]));
+	  insn = emit_insn (gen_divmoddisi3 (operands[3], operands[3],
+					     operands[2]));
+	  REG_NOTES (insn) =
+	    gen_rtx_EXPR_LIST (REG_EQUAL, equal, REG_NOTES (insn));
+	  
+	  insn = emit_move_insn (operands[0], 
+				 gen_lowpart (SImode, operands[3]));
+	  REG_NOTES (insn) =
+	    gen_rtx_EXPR_LIST (REG_EQUAL, 
+			       udiv_equal, REG_NOTES (insn));
         }
     }
   else
@@ -3777,43 +4074,59 @@
       rtx label3 = gen_label_rtx ();
 
       operands[1] = force_reg (SImode, operands[1]);	
+      operands[1] = make_safe_from (operands[1], operands[0]);	
       operands[2] = force_reg (SImode, operands[2]);	
-	
-      emit_move_insn (dr_1, const0_rtx);
+      operands[2] = make_safe_from (operands[2], operands[0]);	
+
+      emit_move_insn (operands[0], const0_rtx);
       emit_insn (gen_cmpsi (operands[2], operands[1]));
       emit_jump_insn (gen_bgtu (label3));
       emit_insn (gen_cmpsi (operands[2], const1_rtx));
       emit_jump_insn (gen_blt (label2));
       emit_insn (gen_cmpsi (operands[2], const1_rtx));
       emit_jump_insn (gen_beq (label1));
-      emit_move_insn (gen_rtx_SUBREG (SImode, tmp, 0), const0_rtx);
-      emit_move_insn (gen_rtx_SUBREG (SImode, tmp, 4), operands[1]);
-      emit_insn (gen_divmoddisi3 (tmp, tmp, operands[2]));
+      emit_insn (gen_zero_extendsidi2 (operands[3], operands[1]));
+      insn = emit_insn (gen_divmoddisi3 (operands[3], operands[3],
+					 operands[2]));
+      REG_NOTES (insn) =
+      gen_rtx_EXPR_LIST (REG_EQUAL, equal, REG_NOTES (insn));
+      
+      insn = emit_move_insn (operands[0], 
+			     gen_lowpart (SImode, operands[3]));
+      REG_NOTES (insn) =
+      gen_rtx_EXPR_LIST (REG_EQUAL, 
+			       udiv_equal, REG_NOTES (insn));
       emit_jump (label3);
       emit_label (label1);
-      emit_move_insn (dr_1, operands[1]);
+      emit_move_insn (operands[0], operands[1]);
       emit_jump (label3);
       emit_label (label2);
-      emit_move_insn (dr_1, const1_rtx);
+      emit_move_insn (operands[0], const1_rtx);
       emit_label (label3);
     }
-
-  emit_move_insn (operands[0], dr_1);
+  emit_move_insn (operands[0], operands[0]);	
   DONE;
 }")
 
 (define_expand "umodsi3"
   [(set (match_operand:SI 0 "register_operand" "=d")
         (umod:SI (match_operand:SI 1 "nonimmediate_operand" "")
-                 (match_operand:SI 2 "nonimmediate_operand" "")))]
+                 (match_operand:SI 2 "nonimmediate_operand" "")))
+   (clobber (match_dup 3))]
   "!TARGET_64BIT"
   "
 {
-  rtx dr_0, dr_1, tmp;
+  rtx insn, udiv_equal, umod_equal, equal;
 
-  tmp = gen_reg_rtx (DImode);
-  dr_0 = gen_rtx_SUBREG (SImode, tmp, 0);	
-  dr_1 = gen_rtx_SUBREG (SImode, tmp, 4);
+  udiv_equal = gen_rtx_UDIV (SImode, operands[1], operands[2]);
+  umod_equal = gen_rtx_UMOD (SImode, operands[1], operands[2]);
+  equal = gen_rtx_IOR (DImode,
+		       gen_rtx_ZERO_EXTEND (DImode, udiv_equal),
+		       gen_rtx_ASHIFT (DImode,
+				       gen_rtx_ZERO_EXTEND (DImode, umod_equal),
+				       GEN_INT (32)));
+
+  operands[3] = gen_reg_rtx (DImode);
 
   if (CONSTANT_P (operands[2]))
     {
@@ -3821,19 +4134,30 @@
         {
           rtx label1 = gen_label_rtx ();
 
-	  emit_move_insn (dr_0, operands[1]);
-          emit_insn (gen_cmpsi (dr_0, operands[2]));
+          operands[1] = make_safe_from (operands[1], operands[0]);
+	  emit_move_insn (operands[0], operands[1]);
+          emit_insn (gen_cmpsi (operands[0], operands[2]));
           emit_jump_insn (gen_bltu (label1));
-	  emit_insn (gen_abssi2 (dr_0, operands[2]));
-          emit_insn (gen_addsi3 (dr_0,dr_0, operands[1]));
+	  emit_insn (gen_abssi2 (operands[0], operands[2]));
+          emit_insn (gen_addsi3 (operands[0], operands[0], operands[1]));
           emit_label (label1);
         }
       else
         {
-          operands[2] = force_const_mem (SImode, operands[2]);
-          emit_move_insn (gen_rtx_SUBREG (SImode, tmp, 0), const0_rtx);
-          emit_move_insn (gen_rtx_SUBREG (SImode, tmp, 4), operands[1]);
-  	  emit_insn (gen_divmoddisi3 (tmp, tmp, operands[2]));
+          operands[2] = force_reg (SImode, operands[2]);	
+          operands[2] = make_safe_from (operands[2], operands[0]);	
+
+	  emit_insn (gen_zero_extendsidi2 (operands[3], operands[1]));
+	  insn = emit_insn (gen_divmoddisi3 (operands[3], operands[3],
+					     operands[2]));
+	  REG_NOTES (insn) =
+	    gen_rtx_EXPR_LIST (REG_EQUAL, equal, REG_NOTES (insn));
+	  
+	  insn = emit_move_insn (operands[0], 
+				 gen_highpart (SImode, operands[3]));
+	  REG_NOTES (insn) =
+	    gen_rtx_EXPR_LIST (REG_EQUAL, 
+			       umod_equal, REG_NOTES (insn));
         }
     }
   else
@@ -3843,28 +4167,36 @@
       rtx label3 = gen_label_rtx ();
 
       operands[1] = force_reg (SImode, operands[1]);	
+      operands[1] = make_safe_from (operands[1], operands[0]);	
       operands[2] = force_reg (SImode, operands[2]);	
+      operands[2] = make_safe_from (operands[2], operands[0]);	
 
-      emit_move_insn(dr_0, operands[1]);	
-      emit_insn (gen_cmpsi (operands[2], dr_0));
+      emit_move_insn(operands[0], operands[1]);	
+      emit_insn (gen_cmpsi (operands[2], operands[1]));
       emit_jump_insn (gen_bgtu (label3));
       emit_insn (gen_cmpsi (operands[2], const1_rtx));
       emit_jump_insn (gen_blt (label2));
       emit_insn (gen_cmpsi (operands[2], const1_rtx));
       emit_jump_insn (gen_beq (label1));
-      emit_move_insn (gen_rtx_SUBREG (SImode, tmp, 0), const0_rtx);
-      emit_move_insn (gen_rtx_SUBREG (SImode, tmp, 4), operands[1]);
-      emit_insn (gen_divmoddisi3 (tmp, tmp, operands[2]));
+      emit_insn (gen_zero_extendsidi2 (operands[3], operands[1]));
+      insn = emit_insn (gen_divmoddisi3 (operands[3], operands[3],
+					 operands[2]));
+      REG_NOTES (insn) =
+      gen_rtx_EXPR_LIST (REG_EQUAL, equal, REG_NOTES (insn));
+      
+      insn = emit_move_insn (operands[0], 
+			     gen_highpart (SImode, operands[3]));
+      REG_NOTES (insn) =
+      gen_rtx_EXPR_LIST (REG_EQUAL, 
+			 umod_equal, REG_NOTES (insn));
       emit_jump (label3);
       emit_label (label1);
-      emit_move_insn (dr_0, const0_rtx);
+      emit_move_insn (operands[0], const0_rtx);
       emit_jump (label3);
       emit_label (label2);
-      emit_insn (gen_subsi3 (dr_0, dr_0, operands[2]));
+      emit_insn (gen_subsi3 (operands[0], operands[0], operands[2]));
       emit_label (label3);
     }
-
-  emit_move_insn (operands[0], dr_0);
   DONE;
 }")
 
@@ -3874,37 +4206,37 @@
 
 (define_expand "divdf3"
   [(parallel
-    [(set (match_operand:DF 0 "nonimmediate_operand" "=f,f")
-          (div:DF (match_operand:DF 1 "general_operand" "0,0")
-                  (match_operand:DF 2 "nonimmediate_operand" "f,m")))
+    [(set (match_operand:DF 0 "register_operand" "=f,f")
+          (div:DF (match_operand:DF 1 "register_operand" "0,0")
+                  (match_operand:DF 2 "general_operand" "f,m")))
      (clobber (reg:CC 33))])]
   "TARGET_HARD_FLOAT"
   "")
 
 (define_insn "*divdf3"
-  [(set (match_operand:DF 0 "nonimmediate_operand" "=f,f")
-        (div:DF (match_operand:DF 1 "general_operand" "0,0")
-                (match_operand:DF 2 "nonimmediate_operand" "f,m")))
+  [(set (match_operand:DF 0 "register_operand" "=f,f")
+        (div:DF (match_operand:DF 1 "register_operand" "0,0")
+                (match_operand:DF 2 "general_operand" "f,m")))
    (clobber (reg:CC 33))]
   "TARGET_HARD_FLOAT && TARGET_IEEE_FLOAT"
   "@
    ddbr\\t%0,%2
    ddb\\t%0,%2"
   [(set_attr "op_type"  "RR,RX")
-   (set_attr "type"    "fdiv")
+   (set_attr "type"     "fdiv")
    (set_attr "atype"    "reg,mem")])
 
 (define_insn "*divdf3_ibm"
-  [(set (match_operand:DF 0 "nonimmediate_operand" "=f,f")
-        (div:DF (match_operand:DF 1 "general_operand" "0,0")
-                (match_operand:DF 2 "nonimmediate_operand" "f,m")))
+  [(set (match_operand:DF 0 "register_operand" "=f,f")
+        (div:DF (match_operand:DF 1 "register_operand" "0,0")
+                (match_operand:DF 2 "general_operand" "f,m")))
    (clobber (reg:CC 33))]
   "TARGET_HARD_FLOAT && TARGET_IBM_FLOAT"
   "@
    ddr\\t%0,%2
    dd\\t%0,%2"
   [(set_attr "op_type"  "RR,RX")
-   (set_attr "type"    "fdiv")
+   (set_attr "type"     "fdiv")
    (set_attr "atype"    "reg,mem")])
 
 ;
@@ -3913,17 +4245,17 @@
 
 (define_expand "divsf3"
   [(parallel
-    [(set (match_operand:SF 0 "nonimmediate_operand" "=f,f")
-          (div:SF (match_operand:SF 1 "nonimmediate_operand" "0,0")
-                  (match_operand:SF 2 "nonimmediate_operand" "f,m")))
+    [(set (match_operand:SF 0 "register_operand" "=f,f")
+          (div:SF (match_operand:SF 1 "register_operand" "0,0")
+                  (match_operand:SF 2 "general_operand" "f,m")))
      (clobber (reg:CC 33))])]
   "TARGET_HARD_FLOAT"
   "")
 
 (define_insn "*divsf3"
-  [(set (match_operand:SF 0 "nonimmediate_operand" "=f,f")
-        (div:SF (match_operand:SF 1 "nonimmediate_operand" "0,0")
-                (match_operand:SF 2 "nonimmediate_operand" "f,m")))
+  [(set (match_operand:SF 0 "register_operand" "=f,f")
+        (div:SF (match_operand:SF 1 "register_operand" "0,0")
+                (match_operand:SF 2 "general_operand" "f,m")))
    (clobber (reg:CC 33))]
   "TARGET_HARD_FLOAT && TARGET_IEEE_FLOAT"
   "@
@@ -3934,9 +4266,9 @@
    (set_attr "atype"    "reg,mem")])
 
 (define_insn "*divsf3"
-  [(set (match_operand:SF 0 "nonimmediate_operand" "=f,f")
-        (div:SF (match_operand:SF 1 "nonimmediate_operand" "0,0")
-                (match_operand:SF 2 "nonimmediate_operand" "f,m")))
+  [(set (match_operand:SF 0 "register_operand" "=f,f")
+        (div:SF (match_operand:SF 1 "register_operand" "0,0")
+                (match_operand:SF 2 "general_operand" "f,m")))
    (clobber (reg:CC 33))]
   "TARGET_HARD_FLOAT && TARGET_IBM_FLOAT"
   "@
@@ -3957,44 +4289,85 @@
 
 (define_insn "*anddi3_cc"
   [(set (reg 33)
-        (compare (and:DI (match_operand:DI 1 "r_or_s_operand" "%0,0,0")
-                         (match_operand:DI 2 "r_or_s_operand" "d,m,Q"))
+        (compare (and:DI (match_operand:DI 1 "register_operand" "%0,0")
+                         (match_operand:DI 2 "general_operand" "d,m"))
                  (const_int 0)))
-   (set (match_operand:DI 0 "r_or_s_operand" "=d,d,Q")
+   (set (match_operand:DI 0 "register_operand" "=d,d")
         (and:DI (match_dup 1) (match_dup 2)))]
   "s390_match_ccmode(insn, CCTmode) && TARGET_64BIT"
   "@
    ngr\\t%0,%2
-   ng\\t%0,%2
-   nc\\t%O0(8,%R0),%2"
-  [(set_attr "op_type"  "RR,RX,SS")
-   (set_attr "atype"    "reg,mem,mem")])
+   ng\\t%0,%2"
+  [(set_attr "op_type"  "RRE,RXE")
+   (set_attr "atype"    "reg,mem")])
 
 (define_insn "*anddi3_cconly"
   [(set (reg 33)
         (compare (and:DI (match_operand:DI 1 "register_operand" "%0,0")
-                         (match_operand:DI 2 "r_or_s_operand" "d,m"))
+                         (match_operand:DI 2 "general_operand" "d,m"))
                  (const_int 0)))
    (clobber (match_scratch:DI 0 "=d,d"))]
   "s390_match_ccmode(insn, CCTmode) && TARGET_64BIT"
   "@
    ngr\\t%0,%2
    ng\\t%0,%2"
-  [(set_attr "op_type"  "RR,RX")
+  [(set_attr "op_type"  "RRE,RXE")
    (set_attr "atype"    "reg,mem")])
 
+(define_insn "*anddi3_ni"
+  [(set (match_operand:DI 0 "register_operand" "=d")
+        (and:DI (match_operand:DI 1 "register_operand" "%0")
+                (match_operand:DI 2 "immediate_operand" "n")))
+   (clobber (reg:CC 33))]
+  "TARGET_64BIT && s390_single_hi (operands[2], DImode, -1) >= 0"
+  "*
+{
+  int part = s390_single_hi (operands[2], DImode, -1);
+  operands[2] = GEN_INT (s390_extract_hi (operands[2], DImode, part));
+
+  switch (part)
+    {
+      case 0: return \"nihh\\t%0,%x2\";
+      case 1: return \"nihl\\t%0,%x2\";
+      case 2: return \"nilh\\t%0,%x2\";
+      case 3: return \"nill\\t%0,%x2\";
+      default: abort ();
+    }
+}"
+  [(set_attr "op_type"  "RI")
+   (set_attr "atype"    "reg")])
+
 (define_insn "anddi3"
-  [(set (match_operand:DI 0 "r_or_s_operand" "=d,d,Q")
-        (and:DI (match_operand:DI 1 "r_or_s_operand" "%0,0,0")
-                (match_operand:DI 2 "r_or_s_operand" "d,m,Q")))
+  [(set (match_operand:DI 0 "register_operand" "=d,d")
+        (and:DI (match_operand:DI 1 "register_operand" "%0,0")
+                (match_operand:DI 2 "general_operand" "d,m")))
    (clobber (reg:CC 33))]
   "TARGET_64BIT"
   "@
    ngr\\t%0,%2
-   ng\\t%0,%2
-   nc\\t%O0(8,%R0),%2"
-  [(set_attr "op_type"  "RR,RX,SS")
-   (set_attr "atype"    "reg,mem,mem")])
+   ng\\t%0,%2"
+  [(set_attr "op_type"  "RRE,RXE")
+   (set_attr "atype"    "reg,mem")])
+
+(define_insn "*anddi3_ss"
+  [(set (match_operand:DI 0 "s_operand" "=Qo")
+        (and:DI (match_dup 0)
+                (match_operand:DI 1 "s_imm_operand" "Qo")))
+   (clobber (reg:CC 33))]
+  ""
+  "nc\\t%O0(8,%R0),%1"
+  [(set_attr "op_type"  "SS")
+   (set_attr "atype"    "mem")])
+
+(define_insn "*anddi3_ss_inv"
+  [(set (match_operand:DI 0 "s_operand" "=Qo")
+        (and:DI (match_operand:DI 1 "s_imm_operand" "Qo")
+                (match_dup 0)))
+   (clobber (reg:CC 33))]
+  ""
+  "nc\\t%O0(8,%R0),%1"
+  [(set_attr "op_type"  "SS")
+   (set_attr "atype"    "mem")])
 
 ;
 ; andsi3 instruction pattern(s).
@@ -4002,23 +4375,22 @@
 
 (define_insn "*andsi3_cc"
   [(set (reg 33)
-        (compare (and:SI (match_operand:SI 1 "r_or_s_operand" "%0,0,0")
-                         (match_operand:SI 2 "r_or_s_operand" "d,m,Q"))
+        (compare (and:SI (match_operand:SI 1 "register_operand" "%0,0")
+                         (match_operand:SI 2 "general_operand" "d,m"))
                  (const_int 0)))
-   (set (match_operand:SI 0 "r_or_s_operand" "=d,d,Q")
+   (set (match_operand:SI 0 "register_operand" "=d,d")
         (and:SI (match_dup 1) (match_dup 2)))]
   "s390_match_ccmode(insn, CCTmode)"
   "@
    nr\\t%0,%2
-   n\\t%0,%2
-   nc\\t%O0(4,%R0),%2"
-  [(set_attr "op_type"  "RR,RX,SS")
-   (set_attr "atype"    "reg,mem,mem")])
+   n\\t%0,%2"
+  [(set_attr "op_type"  "RR,RX")
+   (set_attr "atype"    "reg,mem")])
 
 (define_insn "*andsi3_cconly"
   [(set (reg 33)
         (compare (and:SI (match_operand:SI 1 "register_operand" "%0,0")
-                         (match_operand:SI 2 "r_or_s_operand" "d,m"))
+                         (match_operand:SI 2 "general_operand" "d,m"))
                  (const_int 0)))
    (clobber (match_scratch:SI 0 "=d,d"))]
   "s390_match_ccmode(insn, CCTmode)"
@@ -4028,64 +4400,154 @@
   [(set_attr "op_type"  "RR,RX")
    (set_attr "atype"    "reg,mem")])
 
+(define_insn "*andsi3_ni"
+  [(set (match_operand:SI 0 "register_operand" "=d")
+        (and:SI (match_operand:SI 1 "register_operand" "%0")
+                (match_operand:SI 2 "immediate_operand" "n")))
+   (clobber (reg:CC 33))]
+  "TARGET_64BIT && s390_single_hi (operands[2], SImode, -1) >= 0"
+  "*
+{
+  int part = s390_single_hi (operands[2], SImode, -1);
+  operands[2] = GEN_INT (s390_extract_hi (operands[2], SImode, part));
+
+  switch (part)
+    {
+      case 0: return \"nilh\\t%0,%x2\";
+      case 1: return \"nill\\t%0,%x2\";
+      default: abort ();
+    }
+}"
+  [(set_attr "op_type"  "RI")
+   (set_attr "atype"    "reg")])
+
 (define_insn "andsi3"
-  [(set (match_operand:SI 0 "r_or_s_operand" "=d,d,Q")
-        (and:SI (match_operand:SI 1 "r_or_s_operand" "%0,0,0")
-                (match_operand:SI 2 "r_or_s_operand" "d,m,Q")))
+  [(set (match_operand:SI 0 "register_operand" "=d,d")
+        (and:SI (match_operand:SI 1 "register_operand" "%0,0")
+                (match_operand:SI 2 "general_operand" "d,m")))
    (clobber (reg:CC 33))]
   ""
   "@
    nr\\t%0,%2
-   n\\t%0,%2
-   nc\\t%O0(4,%R0),%2"
-  [(set_attr "op_type"  "RR,RX,SS")
-   (set_attr "atype"    "reg,mem,mem")])
+   n\\t%0,%2"
+  [(set_attr "op_type"  "RR,RX")
+   (set_attr "atype"    "reg,mem")])
+
+(define_insn "*andsi3_ss"
+  [(set (match_operand:SI 0 "s_operand" "=Qo")
+        (and:SI (match_dup 0)
+                (match_operand:SI 1 "s_imm_operand" "Qo")))
+   (clobber (reg:CC 33))]
+  ""
+  "nc\\t%O0(4,%R0),%1"
+  [(set_attr "op_type"  "SS")
+   (set_attr "atype"    "mem")])
+
+(define_insn "*andsi3_ss_inv"
+  [(set (match_operand:SI 0 "s_operand" "=Qo")
+        (and:SI (match_operand:SI 1 "s_imm_operand" "Qo")
+                (match_dup 0)))
+   (clobber (reg:CC 33))]
+  ""
+  "nc\\t%O0(4,%R0),%1"
+  [(set_attr "op_type"  "SS")
+   (set_attr "atype"    "mem")])
 
 ;
 ; andhi3 instruction pattern(s).
 ;
 
-(define_expand "andhi3"
-  [(parallel 
-    [(set (match_operand:HI 0 "r_or_s_operand" "")
-        (and:HI (match_operand:HI 1 "r_or_s_operand" "")
-                (match_operand:HI 2 "r_or_s_operand" "")))
-     (clobber (reg:CC 33))])]
+(define_insn "*andhi3_ni"
+  [(set (match_operand:HI 0 "register_operand" "=d,d")
+        (and:HI (match_operand:HI 1 "register_operand" "%0,0")
+                (match_operand:HI 2 "nonmemory_operand" "d,n")))
+   (clobber (reg:CC 33))]
+  "TARGET_64BIT"
+  "@
+   nr\\t%0,%2
+   nill\\t%0,%x2"
+  [(set_attr "op_type"  "RR,RI")
+   (set_attr "atype"    "reg")])
+
+(define_insn "andhi3"
+  [(set (match_operand:HI 0 "register_operand" "=d")
+        (and:HI (match_operand:HI 1 "register_operand" "%0")
+                (match_operand:HI 2 "nonmemory_operand" "d")))
+   (clobber (reg:CC 33))]
   ""
-  "
-{
-  if (CONSTANT_P (operands[2]))
-      operands[2] = force_const_mem (HImode, operands[2]);
-}")
+  "nr\\t%0,%2"
+  [(set_attr "op_type"  "RR")
+   (set_attr "atype"    "reg")])
+
+(define_insn "*andhi3_ss"
+  [(set (match_operand:HI 0 "s_operand" "=Qo")
+        (and:HI (match_dup 0)
+                (match_operand:HI 1 "s_imm_operand" "Qo")))
+   (clobber (reg:CC 33))]
+  ""
+  "nc\\t%O0(2,%R0),%1"
+  [(set_attr "op_type"  "SS")
+   (set_attr "atype"    "mem")])
 
-(define_insn "*andhi3"
-  [(set (match_operand:HI 0 "r_or_s_operand" "=d,Q")
-        (and:HI (match_operand:HI 1 "r_or_s_operand" "%0,0")
-                (match_operand:HI 2 "r_or_s_operand" "d,Q")))
+(define_insn "*andhi3_ss_inv"
+  [(set (match_operand:HI 0 "s_operand" "=Qo")
+        (and:HI (match_operand:HI 1 "s_imm_operand" "Qo")
+                (match_dup 0)))
    (clobber (reg:CC 33))]
   ""
-  "@
-   nr\\t%0,%2
-   nc\\t%O0(2,%R0),%2"
-  [(set_attr "op_type"  "RR,SS")
-   (set_attr "atype"    "reg,mem")])
+  "nc\\t%O0(2,%R0),%1"
+  [(set_attr "op_type"  "SS")
+   (set_attr "atype"    "mem")])
 
 ;
 ; andqi3 instruction pattern(s).
 ;
 
+(define_insn "*andqi3_ni"
+  [(set (match_operand:QI 0 "register_operand" "=d,d")
+        (and:QI (match_operand:QI 1 "register_operand" "%0,0")
+                (match_operand:QI 2 "nonmemory_operand" "d,n")))
+   (clobber (reg:CC 33))]
+  "TARGET_64BIT"
+  "@
+   nr\\t%0,%2
+   nill\\t%0,%b2"
+  [(set_attr "op_type"  "RR,RI")
+   (set_attr "atype"    "reg")])
+
 (define_insn "andqi3"
-  [(set (match_operand:QI 0 "r_or_s_operand" "=d,Q,Q")
-        (and:QI (match_operand:QI 1 "r_or_s_operand" "%0,0,0")
-                (match_operand:QI 2 "r_or_s_or_im8_operand" "d,n,Q")))
+  [(set (match_operand:QI 0 "register_operand" "=d")
+        (and:QI (match_operand:QI 1 "register_operand" "%0")
+                (match_operand:QI 2 "nonmemory_operand" "d")))
+   (clobber (reg:CC 33))]
+  ""
+  "nr\\t%0,%2"
+  [(set_attr "op_type"  "RR")
+   (set_attr "atype"    "reg")])
+
+(define_insn "*andqi3_ss"
+  [(set (match_operand:QI 0 "s_operand" "=Qo,Qo")
+        (and:QI (match_dup 0)
+                (match_operand:QI 1 "s_imm_operand" "n,Qo")))
    (clobber (reg:CC 33))]
   ""
   "@
-   nr\\t%0,%2
-   ni\\t%0,%b2
-   nc\\t%O0(1,%R0),%2"
-  [(set_attr "op_type"  "RR,SI,SS")
-   (set_attr "atype"    "reg,mem,mem")])
+   ni\\t%0,%b1
+   nc\\t%O0(1,%R0),%1"
+  [(set_attr "op_type"  "SI,SS")
+   (set_attr "atype"    "mem")])
+
+(define_insn "*andqi3_ss_inv"
+  [(set (match_operand:QI 0 "s_operand" "=Qo,Qo")
+        (and:QI (match_operand:QI 1 "s_imm_operand" "n,Qo")
+                (match_dup 0)))
+   (clobber (reg:CC 33))]
+  ""
+  "@
+   ni\\t%0,%b1
+   nc\\t%O0(1,%R0),%1"
+  [(set_attr "op_type"  "SI,SS")
+   (set_attr "atype"    "mem")])
 
 
 ;;
@@ -4096,95 +4558,267 @@
 ; iordi3 instruction pattern(s).
 ;
 
+(define_insn "*iordi3_cc"
+  [(set (reg 33)
+        (compare (ior:DI (match_operand:DI 1 "register_operand" "%0,0")
+                         (match_operand:DI 2 "general_operand" "d,m"))
+                 (const_int 0)))
+   (set (match_operand:DI 0 "register_operand" "=d,d")
+        (ior:DI (match_dup 1) (match_dup 2)))]
+  "s390_match_ccmode(insn, CCTmode) && TARGET_64BIT"
+  "@
+   ogr\\t%0,%2
+   og\\t%0,%2"
+  [(set_attr "op_type"  "RRE,RXE")
+   (set_attr "atype"    "reg,mem")])
+
+(define_insn "*iordi3_cconly"
+  [(set (reg 33)
+        (compare (ior:DI (match_operand:DI 1 "register_operand" "%0,0")
+                         (match_operand:DI 2 "general_operand" "d,m"))
+                 (const_int 0)))
+   (clobber (match_scratch:DI 0 "=d,d"))]
+  "s390_match_ccmode(insn, CCTmode) && TARGET_64BIT"
+  "@
+   ogr\\t%0,%2
+   og\\t%0,%2"
+  [(set_attr "op_type"  "RRE,RXE")
+   (set_attr "atype"    "reg,mem")])
+
+(define_insn "*iordi3_oi"
+  [(set (match_operand:DI 0 "register_operand" "=d")
+        (ior:DI (match_operand:DI 1 "register_operand" "%0")
+                (match_operand:DI 2 "immediate_operand" "n")))
+   (clobber (reg:CC 33))]
+  "TARGET_64BIT && s390_single_hi (operands[2], DImode, 0) >= 0"
+  "*
+{
+  int part = s390_single_hi (operands[2], DImode, 0);
+  operands[2] = GEN_INT (s390_extract_hi (operands[2], DImode, part));
+
+  switch (part)
+    {
+      case 0: return \"oihh\\t%0,%x2\";
+      case 1: return \"oihl\\t%0,%x2\";
+      case 2: return \"oilh\\t%0,%x2\";
+      case 3: return \"oill\\t%0,%x2\";
+      default: abort ();
+    }
+}"
+  [(set_attr "op_type"  "RI")
+   (set_attr "atype"    "reg")])
+
 (define_insn "iordi3"
-  [(set (match_operand:DI 0 "r_or_s_operand" "=d,d,Q,d")
-        (ior:DI (match_operand:DI 1 "r_or_s_operand" "%0,0,0,0")
-                (match_operand:DI 2 "r_or_s_operand" "d,m,Q,L")))
+  [(set (match_operand:DI 0 "register_operand" "=d,d")
+        (ior:DI (match_operand:DI 1 "register_operand" "%0,0")
+                (match_operand:DI 2 "general_operand" "d,m")))
    (clobber (reg:CC 33))]
   "TARGET_64BIT"
   "@
    ogr\\t%0,%2
-   og\\t%0,%2
-   oc\\t%O0(8,%R0),%2
-   oill\\t%0,%2"
-  [(set_attr "op_type"  "RRE,RXE,SS,RI")
-   (set_attr "atype"    "reg,mem,mem,reg")])
+   og\\t%0,%2"
+  [(set_attr "op_type"  "RRE,RXE")
+   (set_attr "atype"    "reg,mem")])
+
+(define_insn "*iordi3_ss"
+  [(set (match_operand:DI 0 "s_operand" "=Qo")
+        (ior:DI (match_dup 0)
+                (match_operand:DI 1 "s_imm_operand" "Qo")))
+   (clobber (reg:CC 33))]
+  ""
+  "oc\\t%O0(8,%R0),%1"
+  [(set_attr "op_type"  "SS")
+   (set_attr "atype"    "mem")])
+
+(define_insn "*iordi3_ss_inv"
+  [(set (match_operand:DI 0 "s_operand" "=Qo")
+        (ior:DI (match_operand:DI 1 "s_imm_operand" "Qo")
+                (match_dup 0)))
+   (clobber (reg:CC 33))]
+  ""
+  "oc\\t%O0(8,%R0),%1"
+  [(set_attr "op_type"  "SS")
+   (set_attr "atype"    "mem")])
 
 ;
 ; iorsi3 instruction pattern(s).
 ;
 
-(define_expand "iorsi3"
-  [(parallel
-    [(set (match_operand:SI 0 "r_or_s_operand" "")
-        (ior:SI (match_operand:SI 1 "r_or_s_operand" "")
-                (match_operand:SI 2 "r_or_s_operand" "")))
-     (clobber (reg:CC 33))])]
-  ""
-  "
+(define_insn "*iorsi3_cc"
+  [(set (reg 33)
+        (compare (ior:SI (match_operand:SI 1 "register_operand" "%0,0")
+                         (match_operand:SI 2 "general_operand" "d,m"))
+                 (const_int 0)))
+   (set (match_operand:SI 0 "register_operand" "=d,d")
+        (ior:SI (match_dup 1) (match_dup 2)))]
+  "s390_match_ccmode(insn, CCTmode)"
+  "@
+   or\\t%0,%2
+   o\\t%0,%2"
+  [(set_attr "op_type"  "RR,RX")
+   (set_attr "atype"    "reg,mem")])
+
+(define_insn "*iorsi3_cconly"
+  [(set (reg 33)
+        (compare (ior:SI (match_operand:SI 1 "register_operand" "%0,0")
+                         (match_operand:SI 2 "general_operand" "d,m"))
+                 (const_int 0)))
+   (clobber (match_scratch:SI 0 "=d,d"))]
+  "s390_match_ccmode(insn, CCTmode)"
+  "@
+   or\\t%0,%2
+   o\\t%0,%2"
+  [(set_attr "op_type"  "RR,RX")
+   (set_attr "atype"    "reg,mem")])
+
+(define_insn "*iorsi3_oi"
+  [(set (match_operand:SI 0 "register_operand" "=d")
+        (ior:SI (match_operand:SI 1 "register_operand" "%0")
+                (match_operand:SI 2 "immediate_operand" "n")))
+   (clobber (reg:CC 33))]
+  "TARGET_64BIT && s390_single_hi (operands[2], SImode, 0) >= 0"
+  "*
 {
-  if (CONSTANT_P (operands[2]))
-      operands[2] =  force_const_mem (SImode, operands[2]);
-}")
+  int part = s390_single_hi (operands[2], SImode, 0);
+  operands[2] = GEN_INT (s390_extract_hi (operands[2], SImode, part));
+
+  switch (part)
+    {
+      case 0: return \"oilh\\t%0,%x2\";
+      case 1: return \"oill\\t%0,%x2\";
+      default: abort ();
+    }
+}"
+  [(set_attr "op_type"  "RI")
+   (set_attr "atype"    "reg")])
 
-(define_insn "*iorsi3"
-  [(set (match_operand:SI 0 "r_or_s_operand" "=d,d,Q")
-        (ior:SI (match_operand:SI 1 "r_or_s_operand" "%0,0,0")
-                (match_operand:SI 2 "r_or_s_operand" "d,m,Q")))
+(define_insn "iorsi3"
+  [(set (match_operand:SI 0 "register_operand" "=d,d")
+        (ior:SI (match_operand:SI 1 "register_operand" "%0,0")
+                (match_operand:SI 2 "general_operand" "d,m")))
    (clobber (reg:CC 33))]
   ""
   "@
    or\\t%0,%2
-   o\\t%0,%2
-   oc\\t%O0(4,%R0),%2"
-  [(set_attr "op_type"  "RR,RX,SS")
-   (set_attr "atype"    "reg,mem,mem")])
+   o\\t%0,%2"
+  [(set_attr "op_type"  "RR,RX")
+   (set_attr "atype"    "reg,mem")])
+
+(define_insn "*iorsi3_ss"
+  [(set (match_operand:SI 0 "s_operand" "=Qo")
+        (ior:SI (match_dup 0)
+                (match_operand:SI 1 "s_imm_operand" "Qo")))
+   (clobber (reg:CC 33))]
+  ""
+  "oc\\t%O0(4,%R0),%1"
+  [(set_attr "op_type"  "SS")
+   (set_attr "atype"    "mem")])
+
+(define_insn "*iorsi3_ss_inv"
+  [(set (match_operand:SI 0 "s_operand" "=Qo")
+        (ior:SI (match_operand:SI 1 "s_imm_operand" "Qo")
+                (match_dup 0)))
+   (clobber (reg:CC 33))]
+  ""
+  "oc\\t%O0(4,%R0),%1"
+  [(set_attr "op_type"  "SS")
+   (set_attr "atype"    "mem")])
 
 ;
 ; iorhi3 instruction pattern(s).
 ;
 
-(define_expand "iorhi3"
-  [(parallel 
-    [(set (match_operand:HI 0 "r_or_s_operand" "")
-        (ior:HI (match_operand:HI 1 "r_or_s_operand" "")
-                (match_operand:HI 2 "r_or_s_operand" "")))
-     (clobber (reg:CC 33))])]
+(define_insn "*iorhi3_oi"
+  [(set (match_operand:HI 0 "register_operand" "=d,d")
+        (ior:HI (match_operand:HI 1 "register_operand" "%0,0")
+                (match_operand:HI 2 "nonmemory_operand" "d,n")))
+   (clobber (reg:CC 33))]
+  "TARGET_64BIT"
+  "@
+   or\\t%0,%2
+   oill\\t%0,%x2"
+  [(set_attr "op_type"  "RR,RI")
+   (set_attr "atype"    "reg")])
+
+(define_insn "iorhi3"
+  [(set (match_operand:HI 0 "register_operand" "=d")
+        (ior:HI (match_operand:HI 1 "register_operand" "%0")
+                (match_operand:HI 2 "nonmemory_operand" "d")))
+   (clobber (reg:CC 33))]
   ""
-  "
-{
-  if (CONSTANT_P (operands[2]))
-      operands[2] =  force_const_mem (HImode, operands[2]);
-}")
+  "or\\t%0,%2"
+  [(set_attr "op_type"  "RR")
+   (set_attr "atype"    "reg")])
+
+(define_insn "*iorhi3_ss"
+  [(set (match_operand:HI 0 "s_operand" "=Qo")
+        (ior:HI (match_dup 0)
+                (match_operand:HI 1 "s_imm_operand" "Qo")))
+   (clobber (reg:CC 33))]
+  ""
+  "oc\\t%O0(2,%R0),%1"
+  [(set_attr "op_type"  "SS")
+   (set_attr "atype"    "mem")])
 
-(define_insn "*iorhi3"
-  [(set (match_operand:HI 0 "r_or_s_operand" "=d,Q")
-        (ior:HI (match_operand:HI 1 "r_or_s_operand" "%0,0")
-                (match_operand:HI 2 "r_or_s_operand" "d,Q")))
+(define_insn "*iorhi3_ss_inv"
+  [(set (match_operand:HI 0 "s_operand" "=Qo")
+        (ior:HI (match_operand:HI 1 "s_imm_operand" "Qo")
+                (match_dup 0)))
    (clobber (reg:CC 33))]
   ""
-  "@
-   or\\t%0,%2
-   oc\\t%O0(2,%R0),%2"
-  [(set_attr "op_type"  "RR,SS")
-   (set_attr "atype"    "reg,mem")])
+  "oc\\t%O0(2,%R0),%1"
+  [(set_attr "op_type"  "SS")
+   (set_attr "atype"    "mem")])
 
 ;
 ; iorqi3 instruction pattern(s).
 ;
 
+(define_insn "*iorqi3_oi"
+  [(set (match_operand:QI 0 "register_operand" "=d,d")
+        (ior:QI (match_operand:QI 1 "register_operand" "%0,0")
+                (match_operand:QI 2 "nonmemory_operand" "d,n")))
+   (clobber (reg:CC 33))]
+  "TARGET_64BIT"
+  "@
+   or\\t%0,%2
+   oill\\t%0,%b2"
+  [(set_attr "op_type"  "RR,RI")
+   (set_attr "atype"    "reg")])
+
 (define_insn "iorqi3"
-  [(set (match_operand:QI 0 "r_or_s_operand" "=d,Q,Q")
-        (ior:QI (match_operand:QI 1 "r_or_s_operand" "%0,0,0")
-                (match_operand:QI 2 "r_or_s_or_im8_operand" "d,n,Q")))
+  [(set (match_operand:QI 0 "register_operand" "=d")
+        (ior:QI (match_operand:QI 1 "register_operand" "%0")
+                (match_operand:QI 2 "nonmemory_operand" "d")))
+   (clobber (reg:CC 33))]
+  ""
+  "or\\t%0,%2"
+  [(set_attr "op_type"  "RR")
+   (set_attr "atype"    "reg")])
+
+(define_insn "*iorqi3_ss"
+  [(set (match_operand:QI 0 "s_operand" "=Qo,Qo")
+        (ior:QI (match_dup 0)
+                (match_operand:QI 1 "s_imm_operand" "n,Qo")))
    (clobber (reg:CC 33))]
   ""
   "@
-   or\\t%0,%2
-   oi\\t%0,%b2
-   oc\\t%O0(1,%R0),%2"
-  [(set_attr "op_type"  "RR,SI,SS")
-   (set_attr "atype"    "reg,mem,mem")])
+   oi\\t%0,%b1
+   oc\\t%O0(1,%R0),%1"
+  [(set_attr "op_type"  "SI,SS")
+   (set_attr "atype"    "reg,mem")])
+
+(define_insn "*iorqi3_ss_inv"
+  [(set (match_operand:QI 0 "s_operand" "=Qo,Qo")
+        (ior:QI (match_operand:QI 1 "s_imm_operand" "n,Qo")
+                (match_dup 0)))
+   (clobber (reg:CC 33))]
+  ""
+  "@
+   oi\\t%0,%b1
+   oc\\t%O0(1,%R0),%1"
+  [(set_attr "op_type"  "SI,SS")
+   (set_attr "atype"    "reg,mem")])
 
 
 ;;
@@ -4195,94 +4829,199 @@
 ; xordi3 instruction pattern(s).
 ;
 
+(define_insn "*xordi3_cc"
+  [(set (reg 33)
+        (compare (xor:DI (match_operand:DI 1 "register_operand" "%0,0")
+                         (match_operand:DI 2 "general_operand" "d,m"))
+                 (const_int 0)))
+   (set (match_operand:DI 0 "register_operand" "=d,d")
+        (xor:DI (match_dup 1) (match_dup 2)))]
+  "s390_match_ccmode(insn, CCTmode) && TARGET_64BIT"
+  "@
+   xgr\\t%0,%2
+   xg\\t%0,%2"
+  [(set_attr "op_type"  "RRE,RXE")
+   (set_attr "atype"    "reg,mem")])
+
+(define_insn "*xordi3_cconly"
+  [(set (reg 33)
+        (compare (xor:DI (match_operand:DI 1 "register_operand" "%0,0")
+                         (match_operand:DI 2 "general_operand" "d,m"))
+                 (const_int 0)))
+   (clobber (match_scratch:DI 0 "=d,d"))]
+  "s390_match_ccmode(insn, CCTmode) && TARGET_64BIT"
+  "@
+   xgr\\t%0,%2
+   xr\\t%0,%2"
+  [(set_attr "op_type"  "RRE,RXE")
+   (set_attr "atype"    "reg,mem")])
+
 (define_insn "xordi3"
-  [(set (match_operand:DI 0 "r_or_s_operand" "=d,d,Q")
-        (xor:DI (match_operand:DI 1 "r_or_s_operand" "%0,0,0")
-                (match_operand:DI 2 "r_or_s_operand" "d,m,Q")))
+  [(set (match_operand:DI 0 "register_operand" "=d,d")
+        (xor:DI (match_operand:DI 1 "register_operand" "%0,0")
+                (match_operand:DI 2 "general_operand" "d,m")))
    (clobber (reg:CC 33))]
   "TARGET_64BIT"
   "@
    xgr\\t%0,%2
-   xg\\t%0,%2
-   xc\\t%O0(8,%R0),%2"
-  [(set_attr "op_type"  "RRE,RXE,SS")
-   (set_attr "atype"    "reg,mem,mem")])
+   xg\\t%0,%2"
+  [(set_attr "op_type"  "RRE,RXE")
+   (set_attr "atype"    "reg,mem")])
+
+(define_insn "*xordi3_ss"
+  [(set (match_operand:DI 0 "s_operand" "=Qo")
+        (xor:DI (match_dup 0)
+                (match_operand:DI 1 "s_imm_operand" "Qo")))
+   (clobber (reg:CC 33))]
+  ""
+  "xc\\t%O0(8,%R0),%1"
+  [(set_attr "op_type"  "SS")
+   (set_attr "atype"    "mem")])
+
+(define_insn "*xordi3_ss_inv"
+  [(set (match_operand:DI 0 "s_operand" "=Qo")
+        (xor:DI (match_operand:DI 1 "s_imm_operand" "Qo")
+                (match_dup 0)))
+   (clobber (reg:CC 33))]
+  ""
+  "xc\\t%O0(8,%R0),%1"
+  [(set_attr "op_type"  "SS")
+   (set_attr "atype"    "mem")])
 
 ;
 ; xorsi3 instruction pattern(s).
 ;
 
-(define_expand "xorsi3"
-  [(parallel
-    [(set (match_operand:SI 0 "r_or_s_operand" "")
-        (xor:SI (match_operand:SI 1 "r_or_s_operand" "")
-                (match_operand:SI 2 "r_or_s_operand" "")))
-     (clobber (reg:CC 33))])]
-  ""
-  "
-{
-  if (CONSTANT_P (operands[2]))
-      operands[2] =  force_const_mem (SImode, operands[2]);
-}")
+(define_insn "*xorsi3_cc"
+  [(set (reg 33)
+        (compare (xor:SI (match_operand:SI 1 "register_operand" "%0,0")
+                         (match_operand:SI 2 "general_operand" "d,m"))
+                 (const_int 0)))
+   (set (match_operand:SI 0 "register_operand" "=d,d")
+        (xor:SI (match_dup 1) (match_dup 2)))]
+  "s390_match_ccmode(insn, CCTmode)"
+  "@
+   xr\\t%0,%2
+   x\\t%0,%2"
+  [(set_attr "op_type"  "RR,RX")
+   (set_attr "atype"    "reg,mem")])
+
+(define_insn "*xorsi3_cconly"
+  [(set (reg 33)
+        (compare (xor:SI (match_operand:SI 1 "register_operand" "%0,0")
+                         (match_operand:SI 2 "general_operand" "d,m"))
+                 (const_int 0)))
+   (clobber (match_scratch:SI 0 "=d,d"))]
+  "s390_match_ccmode(insn, CCTmode)"
+  "@
+   xr\\t%0,%2
+   x\\t%0,%2"
+  [(set_attr "op_type"  "RR,RX")
+   (set_attr "atype"    "reg,mem")])
 
-(define_insn "*xorsi3"
-  [(set (match_operand:SI 0 "r_or_s_operand" "=d,d,Q")
-        (xor:SI (match_operand:SI 1 "r_or_s_operand" "%0,0,0")
-                (match_operand:SI 2 "r_or_s_operand" "d,m,Q")))
+(define_insn "xorsi3"
+  [(set (match_operand:SI 0 "register_operand" "=d,d")
+        (xor:SI (match_operand:SI 1 "register_operand" "%0,0")
+                (match_operand:SI 2 "general_operand" "d,m")))
    (clobber (reg:CC 33))]
   ""
   "@
    xr\\t%0,%2
-   x\\t%0,%2
-   xc\\t%O0(4,%R0),%2"
-  [(set_attr "op_type"  "RR,RX,SS")
-   (set_attr "atype"    "reg,mem,mem")])
+   x\\t%0,%2"
+  [(set_attr "op_type"  "RR,RX")
+   (set_attr "atype"    "reg,mem")])
+
+(define_insn "*xorsi3_ss"
+  [(set (match_operand:SI 0 "s_operand" "=Qo")
+        (xor:SI (match_dup 0)
+                (match_operand:SI 1 "s_imm_operand" "Qo")))
+   (clobber (reg:CC 33))]
+  ""
+  "xc\\t%O0(4,%R0),%1"
+  [(set_attr "op_type"  "SS")
+   (set_attr "atype"    "mem")])
+
+(define_insn "*xorsi3_ss_inv"
+  [(set (match_operand:SI 0 "s_operand" "=Qo")
+        (xor:SI (match_operand:SI 1 "s_imm_operand" "Qo")
+                (match_dup 0)))
+   (clobber (reg:CC 33))]
+  ""
+  "xc\\t%O0(4,%R0),%1"
+  [(set_attr "op_type"  "SS")
+   (set_attr "atype"    "mem")])
 
 ;
 ; xorhi3 instruction pattern(s).
 ;
 
-(define_expand "xorhi3"
-  [(parallel
-    [(set (match_operand:HI 0 "r_or_s_operand" "")
-        (xor:HI (match_operand:HI 1 "r_or_s_operand" "")
-                (match_operand:HI 2 "r_or_s_operand" "")))
-     (clobber (reg:CC 33))])]
+(define_insn "xorhi3"
+  [(set (match_operand:HI 0 "register_operand" "=d")
+        (xor:HI (match_operand:HI 1 "register_operand" "%0")
+                (match_operand:HI 2 "nonmemory_operand" "d")))
+   (clobber (reg:CC 33))]
   ""
-  "
-{
-  if (CONSTANT_P (operands[2]))
-      operands[2] =  force_const_mem (HImode, operands[2]);
-}")
+  "xr\\t%0,%2"
+  [(set_attr "op_type"  "RR")
+   (set_attr "atype"    "reg")])
+
+(define_insn "*xorhi3_ss"
+  [(set (match_operand:HI 0 "s_operand" "=Qo")
+        (xor:HI (match_dup 0)
+                (match_operand:HI 1 "s_imm_operand" "Qo")))
+   (clobber (reg:CC 33))]
+  ""
+  "xc\\t%O0(2,%R0),%1"
+  [(set_attr "op_type"  "SS")
+   (set_attr "atype"    "mem")])
 
-(define_insn "*xorhi3"
-  [(set (match_operand:HI 0 "r_or_s_operand" "=d,Q")
-        (xor:HI (match_operand:HI 1 "r_or_s_operand" "%0,0")
-                (match_operand:HI 2 "r_or_s_operand" "d,Q")))
+(define_insn "*xorhi3_ss_inv"
+  [(set (match_operand:HI 0 "s_operand" "=Qo")
+        (xor:HI (match_operand:HI 1 "s_imm_operand" "Qo")
+                (match_dup 0)))
    (clobber (reg:CC 33))]
   ""
-  "@
-   xr\\t%0,%2
-   xc\\t%O0(2,%R0),%2"
-  [(set_attr "op_type"  "RR,SS")
-   (set_attr "atype"    "reg,mem")])
+  "xc\\t%O0(2,%R0),%1"
+  [(set_attr "op_type"  "SS")
+   (set_attr "atype"    "mem")])
 
 ;
 ; xorqi3 instruction pattern(s).
 ;
 
 (define_insn "xorqi3"
-  [(set (match_operand:QI 0 "r_or_s_operand" "=d,Q,Q")
-        (xor:QI (match_operand:QI 1 "r_or_s_operand" "0,0,0")
-                (match_operand:QI 2 "r_or_s_or_im8_operand" "d,n,Q")))
+  [(set (match_operand:QI 0 "register_operand" "=d")
+        (xor:QI (match_operand:QI 1 "register_operand" "%0")
+                (match_operand:QI 2 "nonmemory_operand" "d")))
+   (clobber (reg:CC 33))]
+  ""
+  "xr\\t%0,%2"
+  [(set_attr "op_type"  "RR")
+   (set_attr "atype"    "reg")])
+
+(define_insn "*xorqi3_ss"
+  [(set (match_operand:QI 0 "s_operand" "=Qo,Qo")
+        (xor:QI (match_dup 0)
+                (match_operand:QI 1 "s_imm_operand" "n,Qo")))
    (clobber (reg:CC 33))]
   ""
   "@
-   xr\\t%0,%2
-   xi\\t%0,%b2
-   xc\\t%O0(1,%R0),%2"
-  [(set_attr "op_type"  "RR,SI,SS")
-   (set_attr "atype"    "reg,mem,mem")])
+   xi\\t%0,%b1
+   xc\\t%O0(1,%R0),%1"
+  [(set_attr "op_type"  "SI,SS")
+   (set_attr "atype"    "mem")])
+
+(define_insn "*xorqi3_ss_inv"
+  [(set (match_operand:QI 0 "s_operand" "=Qo,Qo")
+        (xor:QI (match_operand:QI 1 "s_imm_operand" "n,Qo")
+                (match_dup 0)))
+   (clobber (reg:CC 33))]
+  ""
+  "@
+   xi\\t%0,%b1
+   xc\\t%O0(1,%R0),%1"
+  [(set_attr "op_type"  "SI,SS")
+   (set_attr "atype"    "mem")])
 
 
 ;;
@@ -4327,7 +5066,8 @@
   return \"\";
 }"
   [(set_attr "op_type"  "NN")
-   (set_attr "length"  "10")])
+   (set_attr "type"     "other")
+   (set_attr "length"   "10")])
 
 ;
 ; negsi2 instruction pattern(s).
@@ -4427,20 +5167,6 @@
   [(set_attr "op_type"  "RR")])
 
 ;
-; abshi2 instruction pattern(s).
-;
-
-(define_insn "abshi2"
-  [(set (match_operand:HI 0 "register_operand" "=d")
-        (abs:HI (match_operand:HI 1 "register_operand" "d")))
-   (clobber (reg:CC 33))]
-  ""
-  "sll\\t%1,16\;sra\\t%1,16\;lpr\\t%0,%1"
-  [(set_attr "op_type"  "NN")
-   (set_attr "type"    "o3")
-   (set_attr "length"   "10")])
-
-;
 ; absdf2 instruction pattern(s).
 ;
 
@@ -4496,6 +5222,35 @@
   "lper\\t%0,%1"
   [(set_attr "op_type"  "RR")])
 
+;;
+;;- Square root instructions.
+;;
+
+;
+; sqrtdf2 instruction pattern(s).
+;
+
+(define_insn "sqrtdf2"
+  [(set (match_operand:DF 0 "register_operand" "=f,f")
+	(sqrt:DF (match_operand:DF 1 "general_operand" "f,m")))]
+  "TARGET_HARD_FLOAT && TARGET_IEEE_FLOAT"
+  "@
+   sqdbr\\t%0,%1
+   sqdb\\t%0,%1"
+  [(set_attr "op_type"  "RRE,RSE")])
+
+;
+; sqrtsf2 instruction pattern(s).
+;
+
+(define_insn "sqrtsf2"
+  [(set (match_operand:SF 0 "register_operand" "=f,f")
+	(sqrt:SF (match_operand:SF 1 "general_operand" "f,m")))]
+  "TARGET_HARD_FLOAT && TARGET_IEEE_FLOAT"
+  "@
+   sqebr\\t%0,%1
+   sqeb\\t%0,%1"
+  [(set_attr "op_type"  "RRE,RSE")])
 
 ;;
 ;;- One complement instructions.
@@ -4504,89 +5259,54 @@
 ;
 ; one_cmpldi2 instruction pattern(s).
 ;
-
+ 
 (define_expand "one_cmpldi2"
   [(parallel
-    [(set (match_operand:DI 0 "r_or_s_operand" "=d")
-          (not:DI (match_operand:DI 1 "r_or_s_operand" "0")))
-     (use (match_dup 2))
+    [(set (match_operand:DI 0 "register_operand" "")
+          (xor:DI (match_operand:DI 1 "register_operand" "")
+                  (const_int -1)))
      (clobber (reg:CC 33))])]
   "TARGET_64BIT"
-  "{ operands[2] = force_const_mem (DImode, constm1_rtx); }")
-
-(define_insn "*one_cmpldi2"
-  [(set (match_operand:DI 0 "r_or_s_operand" "=d,Q")
-        (not:DI (match_operand:DI 1 "r_or_s_operand" "0,0")))
-   (use (match_operand:DI 2 "memory_operand" "m,m"))
-   (clobber (reg:CC 33))]
-  ""
-  "@
-   xg\\t%0,%2
-   xc\\t%O0(8,%R0),%2"
-  [(set_attr "op_type"  "RR,SS")
-   (set_attr "atype"    "mem")])
-
+  "")
+ 
 ;
 ; one_cmplsi2 instruction pattern(s).
 ;
-
+ 
 (define_expand "one_cmplsi2"
   [(parallel
-    [(set (match_operand:SI 0 "r_or_s_operand" "=d")
-          (not:SI (match_operand:SI 1 "r_or_s_operand" "0")))
-     (use (match_dup 2))
+    [(set (match_operand:SI 0 "register_operand" "")
+          (xor:SI (match_operand:SI 1 "register_operand" "")
+                  (const_int -1)))
      (clobber (reg:CC 33))])]
   ""
-  "{ operands[2] = force_const_mem (SImode, constm1_rtx); }")
-
-(define_insn "*one_cmplsi2"
-  [(set (match_operand:SI 0 "r_or_s_operand" "=d,Q")
-        (not:SI (match_operand:SI 1 "r_or_s_operand" "0,0")))
-   (use (match_operand:SI 2 "memory_operand" "m,m"))
-   (clobber (reg:CC 33))]
-  ""
-  "@
-   x\\t%0,%2
-   xc\\t%O0(4,%R0),%2"
-  [(set_attr "op_type"  "RR,SS")
-   (set_attr "atype"    "mem")])
-
+  "")
+ 
 ;
 ; one_cmplhi2 instruction pattern(s).
 ;
-
+ 
 (define_expand "one_cmplhi2"
   [(parallel
-    [(set (match_operand:HI 0 "r_or_s_operand" "=d")
-          (not:HI (match_operand:HI 1 "r_or_s_operand" "0")))
-     (use (match_dup 2))
+    [(set (match_operand:HI 0 "register_operand" "")
+          (xor:HI (match_operand:HI 1 "register_operand" "")
+                  (const_int -1)))
      (clobber (reg:CC 33))])]
   ""
-  "{ operands[2] = force_const_mem (SImode, constm1_rtx); }")
-
-(define_insn "*one_cmplhi2"
-  [(set (match_operand:HI 0 "r_or_s_operand" "=d,Q")
-        (not:HI (match_operand:HI 1 "r_or_s_operand" "0,0")))
-   (use (match_operand:SI 2 "memory_operand" "m,m"))
-   (clobber (reg:CC 33))]
-  ""
-  "@
-   x\\t%0,%2
-   xc\\t%O0(2,%R0),%2"
-  [(set_attr "op_type"  "RX,SS")
-   (set_attr "atype"    "mem")])
-
+  "")
+ 
 ;
 ; one_cmplqi2 instruction pattern(s).
 ;
-
-(define_insn "one_cmpqi2"
-  [(set (match_operand:QI 0 "memory_operand" "=Q")
-        (not:QI (match_operand:QI 1 "memory_operand" "0")))
-   (clobber (reg:CC 33))]
+ 
+(define_expand "one_cmplqi2"
+  [(parallel
+    [(set (match_operand:QI 0 "register_operand" "")
+          (xor:QI (match_operand:QI 1 "register_operand" "")
+                  (const_int -1)))
+     (clobber (reg:CC 33))])]
   ""
-  "xi\\t%0,255"
-  [(set_attr "op_type"  "SI")])
+  "")
 
 
 ;;
@@ -4707,7 +5427,7 @@
 (define_insn "ashlsi3"
   [(set (match_operand:SI 0 "register_operand" "=d,d")
         (ashift:SI (match_operand:SI 1 "register_operand" "0,0")
-                   (match_operand:SI 2 "r_or_im8_operand" "J,a")))
+                   (match_operand:SI 2 "nonmemory_operand" "J,a")))
    (clobber (reg:CC 33))]
   ""
   "@
@@ -4722,7 +5442,7 @@
 (define_insn "ashrsi3"
   [(set (match_operand:SI 0 "register_operand" "=d,d")
         (ashiftrt:SI (match_operand:SI 1 "register_operand" "0,0")
-                     (match_operand:SI 2 "r_or_im8_operand" "J,a")))
+                     (match_operand:SI 2 "nonmemory_operand" "J,a")))
    (clobber (reg:CC 33))]
   ""
   "@
@@ -4730,37 +5450,6 @@
    sra\\t%0,0(%2)"
   [(set_attr "op_type"  "RS")])
 
-;
-; ashlhi3 instruction pattern(s).
-;
-
-(define_insn "ashlhi3"
-  [(set (match_operand:HI 0 "register_operand" "=d,d")
-        (ashift:HI (match_operand:HI 1 "register_operand" "0,0")
-                   (match_operand:SI 2 "r_or_im8_operand" "J,a")))
-   (clobber (reg:CC 33))]
-  ""
-  "@
-   sll\\t%0,%c2
-   sll\\t%0,0(%2)"
-  [(set_attr "op_type"  "RS,RS")])
-
-;
-; ashrhi3 instruction pattern(s).
-;
-
-(define_insn "ashrhi3"
-  [(set (match_operand:HI 0 "register_operand" "=d,d")
-        (ashiftrt:HI (match_operand:HI 1 "register_operand" "0,0")
-                     (match_operand:SI 2 "r_or_im8_operand" "J,a")))
-   (clobber (reg:CC 33))]
-  ""
-  "@
-   sll\\t%0,16\;sra\\t%0,16+%c2
-   sll\\t%0,16\;sra\\t%0,16(%2)"
-   [(set_attr "op_type"  "NN,NN")
-    (set_attr "length"   "8,8")])
-
 
 ;;
 ;;- logical shift instructions.
@@ -4808,7 +5497,7 @@
 (define_insn "lshrsi3"
   [(set (match_operand:SI 0 "register_operand" "=d,d")
         (lshiftrt:SI (match_operand:SI 1 "register_operand" "0,0")
-                     (match_operand:SI 2 "r_or_im8_operand" "J,a")))
+                     (match_operand:SI 2 "nonmemory_operand" "J,a")))
    (clobber (reg:CC 33))]
   ""
   "@
@@ -4816,22 +5505,6 @@
    srl\\t%0,0(%2)"
   [(set_attr "op_type"  "RS")])
 
-;
-; lshrhi3 instruction pattern(s).
-;
-
-(define_insn "lshrhi3"
-  [(set (match_operand:HI 0 "register_operand" "=d,d")
-        (lshiftrt:HI (match_operand:HI 1 "register_operand" "0,0")
-                     (match_operand:SI 2 "r_or_im8_operand" "J,a")))
-   (clobber (reg:CC 33))]
-  ""
-  "@
-   sll\\t%0,16\;srl\\t%0,16+%c2
-   sll\\t%0,16\;srl\\t%0,16(%2)"
-  [(set_attr "op_type"  "NN,NN")
-   (set_attr "length"   "8,8")])
-
 
 ;;
 ;; Branch instruction patterns.
@@ -5280,7 +5953,9 @@
 (define_insn "blockage"
   [(unspec_volatile [(const_int 0)] 0)]
   ""
-  "")
+  ""
+  [(set_attr "type"    "none")])
+
 
 
 ;
@@ -5336,7 +6011,7 @@
   "TARGET_64BIT"
   "brasl\\t%2,%0"
   [(set_attr "op_type" "RIL")
-   (set_attr "type"   "jsr")])
+   (set_attr "type"    "jsr")])
 
 (define_insn "bras"
   [(call (mem:QI (match_operand:SI 0 "bras_sym_operand" "X"))
@@ -5345,7 +6020,7 @@
   "TARGET_SMALL_EXEC"
   "bras\\t%2,%0"
   [(set_attr "op_type" "RI")
-   (set_attr "type"   "jsr")])
+   (set_attr "type"    "jsr")])
 
 (define_insn "basr_64"
   [(call (mem:QI (match_operand:DI 0 "register_operand" "a"))
@@ -5688,7 +6363,7 @@
     return \"basr\\t13,0\;ahi\\t13,%Y0\";
 }"
   [(set_attr "op_type" "NN")
-   (set_attr "type"    "o2")
+   (set_attr "type"    "la")
    (set_attr "length"  "8")])
 
 (define_insn "ltorg"
@@ -5701,9 +6376,64 @@
    return \"0:\";
 }"
   [(set_attr "op_type" "NN")
-   (set_attr "type"    "branch")
+   (set_attr "type"    "other")
    (set_attr "length"  "4096")])
 
+;;
+;; Insns related to generating the function prologue and epilogue.
+;;
+
+
+(define_expand "prologue"
+  [(use (const_int 0))]
+  ""
+  "
+{
+      s390_emit_prologue ();
+      DONE;
+}")
+
+(define_expand "epilogue"
+  [(use (const_int 1))]
+  ""
+  "
+{
+      s390_emit_epilogue ();
+      DONE;
+}")
+
+
+(define_insn "*return_si"
+  [(return)
+   (use (match_operand:SI 0 "register_operand" "a"))]
+  "!TARGET_64BIT"
+  "br\\t%0"
+  [(set_attr "op_type" "RR")
+   (set_attr "type"    "jsr")		
+   (set_attr "atype"   "mem")])
+
+(define_insn "*return_di"
+  [(return)
+   (use (match_operand:DI 0 "register_operand" "a"))]
+  "TARGET_64BIT"
+  "br\\t%0"
+  [(set_attr "op_type" "RR")
+   (set_attr "type"    "jsr")		
+   (set_attr "atype"   "mem")])
+
+
+(define_insn "lit"
+  [(set (reg 13) (pc))
+   (unspec_volatile [(const_int 0)] 200)]
+  ""
+  "*
+{
+   s390_output_constant_pool (asm_out_file);
+   return \"\";
+}"
+  [(set_attr "op_type" "NN")
+   (set_attr "type"    "integer")])
+
 
 ;;
 ;; Peephole optimization patterns.
diff --git a/gcc/config/s390/t-linux b/gcc/config/s390/t-linux
index 91585b7..bc8d519 100644
--- a/gcc/config/s390/t-linux
+++ b/gcc/config/s390/t-linux
@@ -1,5 +1,6 @@
 # The crtbegin and crtend must not depend on a small GOT
 CRTSTUFF_T_CFLAGS = -O2 -fPIC
+CRTSTUFF_T_CFLAGS_S = -O2 -fPIC
 
 # Compile libgcc2.a with pic.
 TARGET_LIBGCC2_CFLAGS = -fPIC -include $(srcdir)/config/s390/fixdfdi.h