Fix MMIX breakage; ICE in df_ref_record, at df-scan.c:2598

This bug made me dive into some of the murkier waters of gcc, namely
the source of operand 2 to the "call" pattern.  It can be pretty
poisonous, but is unused (either directly or later) by most targets.

The target function_arg (and function_incoming_arg), can unless
specially handled, cause a VOIDmode reg RTX to be generated, for the
function arguments end-marker.  This is then passed on by expand_call
to the target "call" pattern, as operand[2] (which is wrongly
documented or wrongly implemented, see comment in mmix.c) but unused
by most targets that do not handle it specially, as in operand 2 not
making it into the insn generated for the "call" (et al) patterns.  Of
course, the MMIX port stands out here: the RTX makes it into the
generated RTX but is then actually unused and is just a placeholder;
see mmix_print_operand 'p'.

Anyway, df-scan inspects the emitted call rtx and horks on the
void-mode RTX (actually: that it represents a zero-sized register
range) from r12-1702.

While I could replace or remove the emitted unused call insn operand,
that would still leave unusable rtx to future users of function_arg
actually looking for next_arg_reg.  Better replace VOIDmode with
DImode here; that's the "natural" mode of MMIX registers.

(As a future improvement, I'll also remove the placeholder argument
and replace the intended user; the print_operand output modifier 'p'
modifier (as in "PUSHJ $%p2,%0") with some punctuation, perhaps '!'
(as in "PUSHJ $%!,%0").

I inspected all ports, but other targets emit a special
function_arg_info::end_marker cookie or just don't emit "call"
operand[2] (etc) in the expanded "call" pattern.

gcc:
	* config/mmix/mmix.c (mmix_function_arg_1): Avoid
	generating a VOIDmode register for e.g the
	function_arg_info::end_marker.

1 files changed, 10 insertions, 3 deletions

diff --git a/gcc/config/mmix/mmix.c b/gcc/config/mmix/mmix.c
index 40bfb43..db7af7b 100644
--- a/gcc/config/mmix/mmix.c
+++ b/gcc/config/mmix/mmix.c
@@ -667,10 +667,17 @@ mmix_function_arg_1 (const cumulative_args_t argsp_v,
 {
   CUMULATIVE_ARGS *argsp = get_cumulative_args (argsp_v);
 
+  /* The mode of the argument will be VOIDmode for the "end_marker".  Make sure
+     we don't ever generate a VOIDmode register; later passes will barf on that.
+     We may want to use the register number, so return something nominally
+     useful.  Thus, for VOIDmode, use DImode, being the natural mode for the
+     register.  */
+  machine_mode mode = arg.mode == VOIDmode ? DImode : arg.mode;
+
   /* Last-argument marker.  */
   if (arg.end_marker_p ())
     return (argsp->regs < MMIX_MAX_ARGS_IN_REGS)
-      ? gen_rtx_REG (arg.mode,
+      ? gen_rtx_REG (mode,
 		     (incoming
 		      ? MMIX_FIRST_INCOMING_ARG_REGNUM
 		      : MMIX_FIRST_ARG_REGNUM) + argsp->regs)
@@ -678,10 +685,10 @@ mmix_function_arg_1 (const cumulative_args_t argsp_v,
 
   return (argsp->regs < MMIX_MAX_ARGS_IN_REGS
 	  && !targetm.calls.must_pass_in_stack (arg)
-	  && (GET_MODE_BITSIZE (arg.mode) <= 64
+	  && (GET_MODE_BITSIZE (mode) <= 64
 	      || argsp->lib
 	      || TARGET_LIBFUNC))
-    ? gen_rtx_REG (arg.mode,
+    ? gen_rtx_REG (mode,
 		   (incoming
 		    ? MMIX_FIRST_INCOMING_ARG_REGNUM
 		    : MMIX_FIRST_ARG_REGNUM)