PowerPC instruction mask checks

The instruction mask bits should never overlap any of the operands,
nor should operand bits overlap, but some operands weren't checked.
This patch arranges to check the omitted operands, using a mask
returned by the operand->insert function.  Some tweaking of various
insert functions is needed to support this: The error case must set
field bits.

Since I was looking at the insert functions, I tidied some dead code
and simplified some of the powerpc_operands entries.

gas/
	* config/tc-ppc.c (insn_validate): Don't ignore mask in
	PPC_OPSHIFT_INV case.  Call the insert function to calculate
	a mask.
opcodes/
	* ppc-opc.c (insert_arx, insert_ary, insert_rx, insert_ry, insert_ls),
	(insert_evuimm1_ex0, insert_evuimm2_ex0, insert_evuimm4_ex0),
	(insert_evuimm8_ex0, insert_evuimm_lt8, insert_evuimm_lt16),
	(insert_rD_rS_even, insert_off_lsp, insert_off_spe2, insert_Ddd):
	Don't return zero on error, insert mask bits instead.
	(insert_sd4h, extract_sd4h, insert_sd4w, extract_sd4w): Delete.
	(insert_sh6, extract_sh6): Delete dead code.
	(insert_sprbat, insert_sprg): Use unsigned comparisions.
	(powerpc_operands <OIMM>): Set shift count rather than using
	PPC_OPSHIFT_INV.
	<SE_SDH, SE_SDW>: Likewise.  Don't use insert/extract functions.

2 files changed, 29 insertions, 14 deletions

diff --git a/gas/ChangeLog b/gas/ChangeLog
index 2773d92..601f2aa 100644
--- a/gas/ChangeLog
+++ b/gas/ChangeLog
@@ -1,5 +1,11 @@
 2018-11-06  Alan Modra  <amodra@gmail.com>
 
+	* config/tc-ppc.c (insn_validate): Don't ignore mask in
+	PPC_OPSHIFT_INV case.  Call the insert function to calculate
+	a mask.
+
+2018-11-06  Alan Modra  <amodra@gmail.com>
+
 	* config/tc-ppc.c (insn_validate): Check that optional operands
 	are not followed by non-optional operands.
 
diff --git a/gas/config/tc-ppc.c b/gas/config/tc-ppc.c
index 694c47a..9a06668 100644
--- a/gas/config/tc-ppc.c
+++ b/gas/config/tc-ppc.c
@@ -1522,23 +1522,32 @@ insn_validate (const struct powerpc_opcode *op)
         }
       else
         {
+	  uint64_t mask;
 	  const struct powerpc_operand *operand = &powerpc_operands[*o];
-	  if (operand->shift != (int) PPC_OPSHIFT_INV)
+	  if (operand->shift == (int) PPC_OPSHIFT_INV)
 	    {
-	      uint64_t mask;
-
-	      if (operand->shift >= 0)
-		mask = operand->bitm << operand->shift;
-	      else
-		mask = operand->bitm >> -operand->shift;
-	      if (omask & mask)
-		{
-		  as_bad (_("operand %d overlap in %s"),
-			  (int) (o - op->operands), op->name);
-		  return TRUE;
-		}
-	      omask |= mask;
+	      const char *errmsg;
+	      int64_t val;
+
+	      errmsg = NULL;
+	      val = -1;
+	      if ((operand->flags & PPC_OPERAND_NEGATIVE) != 0)
+		val = -val;
+	      else if ((operand->flags & PPC_OPERAND_PLUS1) != 0)
+		val += 1;
+	      mask = (*operand->insert) (0, val, ppc_cpu, &errmsg);
+	    }
+	  else if (operand->shift >= 0)
+	    mask = operand->bitm << operand->shift;
+	  else
+	    mask = operand->bitm >> -operand->shift;
+	  if (omask & mask)
+	    {
+	      as_bad (_("operand %d overlap in %s"),
+		      (int) (o - op->operands), op->name);
+	      return TRUE;
 	    }
+	  omask |= mask;
 	  if ((operand->flags & PPC_OPERAND_OPTIONAL) != 0)
 	    optional = TRUE;
 	  else if (optional)