MIPS/Linux: Disable n32 USR `ptrace' accesses to 64-bit registers

On the MIPS target DSP ASE registers can only be accessed with the
PTRACE_PEEKUSR and PTRACE_POKEUSR `ptrace' requests.  With the n32 ABI
these requests only pass 32-bit data quantities, which are narrower than
the width of DSP accumulator registers, which are 64-bit.

Generic code is prepared to transfer registers wider than the `ptrace'
data type by offsetting into the USR address space, by the data width
transferred.  That however does not work with the MIPS target, because
of how the API has been defined, where USR register addresses are
actually indices rather than offsets.  Consequently given address `a'
using `a + 4' accesses the fourth next register rather than the upper
half of the original register.

With native debugging this causes clobbered register contents, as well
as access failures as locations beyond the available USR space are
addressed:

(gdb) info registers
                  zero               at               v0               v1
 R0   0000000000000000 0000000000000001 0000000000000001 0000000000000000
                    a0               a1               a2               a3
 R4   0000000010019158 0000000000000000 0000000000000011 0000000010019160
                    a4               a5               a6               a7
 R8   0000000010019160 fffffffffff00000 fffffffffffffff8 0000000000000000
                    t0               t1               t2               t3
 R12  0000000010019150 0000000000000001 0000000000000001 000000000000000f
                    s0               s1               s2               s3
 R16  0000000077ee6f20 0000000010007bb0 0000000000000000 0000000000000000
                    s4               s5               s6               s7
 R20  000000000052e668 000000000052f008 0000000000000000 0000000000000000
                    t8               t9               k0               k1
 R24  0000000000000001 0000000010019010 0000000000000000 0000000000000000
                    gp               sp               s8               ra
 R28  0000000010020280 000000007fff4c10 000000007fff4c10 0000000010004f48
                status               lo               hi         badvaddr
      0000000000109cf3 0000000000943efe 000000000000000e 000000001001900c
                 cause               pc
      0000000000800024 0000000010004f48
                  fcsr              fir              hi1              lo1
              0e800000         00f30000 0000000004040404 0101010105050505
                   hi2              lo2              hi3              lo3
      0202020255aa33cc Couldn't read register  (#75): Input/output error.
(gdb)

With `gdbserver' this makes debugging impossible due to a fatal failure:

(gdb) target remote :2346
Remote debugging using :2346
Reading symbols from .../sysroot/mips-r2-hard/lib32/ld.so.1...done.
0x77fc3d50 in __start () from .../sysroot/mips-r2-hard/lib32/ld.so.1
(gdb) continue
Continuing.
warning: Remote failure reply: E01
Remote communication error.  Target disconnected.: Connection reset by peer.
(gdb)

Correct the problem by marking any register in the MIPS backend whose
width exceeds the width of the `ptrace' data type unavailable for the
purpose of PTRACE_PEEKUSR and PTRACE_POKEUSR requests:

(gdb) info registers
                  zero               at               v0               v1
 R0   0000000000000000 0000000000000001 0000000000000001 0000000000000000
                    a0               a1               a2               a3
 R4   0000000010019158 0000000000000000 0000000000000011 0000000010019160
                    a4               a5               a6               a7
 R8   0000000010019160 fffffffffff00000 fffffffffffffff8 0000000000000000
                    t0               t1               t2               t3
 R12  0000000010019150 0000000000000001 0000000000000001 000000000000000f
                    s0               s1               s2               s3
 R16  0000000077ee6f20 0000000010007bb0 0000000000000000 0000000000000000
                    s4               s5               s6               s7
 R20  000000000052e5c8 000000000052f008 0000000000000000 0000000000000000
                    t8               t9               k0               k1
 R24  0000000000000001 0000000010019010 0000000000000000 0000000000000000
                    gp               sp               s8               ra
 R28  0000000010020280 000000007fff4be0 000000007fff4be0 0000000010004f48
                status               lo               hi         badvaddr
      0000000000109cf3 0000000000943efe 000000000000000e 000000001001900c
                 cause               pc
      0000000000800024 0000000010004f48
                  fcsr              fir              hi1              lo1
              0e800000         00f30000    <unavailable>    <unavailable>
                   hi2              lo2              hi3              lo3
         <unavailable>    <unavailable>    <unavailable>    <unavailable>
                dspctl          restart
              55aa33cc 0000000000000000
(gdb)

as there is no way to access full contents of these registers with the
limited API available anyway.

This obviously does not affect general-purpose registers (which use the
PTRACE_GETREGS and PTRACE_SETREGS requests for access) or floating-point
general registers (which use PTRACE_GETFPREGS and PTRACE_SETFPREGS).
And $dspctl, being 32-bit, remains accessible too, which is important
for BPOSGE32 branch decoding in single-stepping.

For DSP accumulator access with the n32 ABI a new `ptrace' API is required
on the kernel side.

	gdb/
	* mips-linux-nat.c (mips64_linux_register_addr): Return -1 if
	the width of the requested register exceeds the width of the
	`ptrace' data type.

	gdb/gdbserver/
	* linux-mips-low.c (mips_cannot_fetch_register): Return 1 if the
	width of the requested register exceeds the width of the
	`ptrace' data type.
	(mips_cannot_store_register): Likewise.

4 files changed, 26 insertions, 0 deletions

diff --git a/gdb/ChangeLog b/gdb/ChangeLog
index 82e249f..8d425a7 100644
--- a/gdb/ChangeLog
+++ b/gdb/ChangeLog
@@ -1,3 +1,9 @@
+2018-05-22  Maciej W. Rozycki  <macro@mips.com>
+
+	* mips-linux-nat.c (mips64_linux_register_addr): Return -1 if
+	the width of the requested register exceeds the width of the
+	`ptrace' data type.
+
 2018-05-21  Tom Tromey  <tom@tromey.com>
 
 	* printcmd.c (output_command): Remove.
diff --git a/gdb/gdbserver/ChangeLog b/gdb/gdbserver/ChangeLog
index 3c55457..c577b3d 100644
--- a/gdb/gdbserver/ChangeLog
+++ b/gdb/gdbserver/ChangeLog
@@ -1,3 +1,10 @@
+2018-05-22  Maciej W. Rozycki  <macro@mips.com>
+
+	* linux-mips-low.c (mips_cannot_fetch_register): Return 1 if the
+	width of the requested register exceeds the width of the
+	`ptrace' data type.
+	(mips_cannot_store_register): Likewise.
+
 2018-05-21  Maciej W. Rozycki  <macro@mips.com>
 
 	* linux-mips-low.c (mips_fetch_register): New function.  Update
diff --git a/gdb/gdbserver/linux-mips-low.c b/gdb/gdbserver/linux-mips-low.c
index ff87dd7..7fae2e6 100644
--- a/gdb/gdbserver/linux-mips-low.c
+++ b/gdb/gdbserver/linux-mips-low.c
@@ -211,6 +211,10 @@ mips_cannot_fetch_register (int regno)
 
   tdesc = current_process ()->tdesc;
 
+  /* On n32 we can't access 64-bit registers via PTRACE_PEEKUSR.  */
+  if (register_size (tdesc, regno) > sizeof (PTRACE_XFER_TYPE))
+    return 1;
+
   if (find_regno (tdesc, "r0") == regno)
     return 1;
 
@@ -227,6 +231,10 @@ mips_cannot_store_register (int regno)
 
   tdesc = current_process ()->tdesc;
 
+  /* On n32 we can't access 64-bit registers via PTRACE_POKEUSR.  */
+  if (register_size (tdesc, regno) > sizeof (PTRACE_XFER_TYPE))
+    return 1;
+
   if (find_regno (tdesc, "r0") == regno)
     return 1;
 
diff --git a/gdb/mips-linux-nat.c b/gdb/mips-linux-nat.c
index b1dbb79..a9f0b79 100644
--- a/gdb/mips-linux-nat.c
+++ b/gdb/mips-linux-nat.c
@@ -145,6 +145,11 @@ mips64_linux_register_addr (struct gdbarch *gdbarch, int regno, int store)
   if (regno < 0 || regno >= gdbarch_num_regs (gdbarch))
     error (_("Bogon register number %d."), regno);
 
+  /* On n32 we can't access 64-bit registers via PTRACE_PEEKUSR
+     or PTRACE_POKEUSR.  */
+  if (register_size (gdbarch, regno) > sizeof (PTRACE_TYPE_RET))
+    return (CORE_ADDR) -1;
+
   if (regno > MIPS_ZERO_REGNUM && regno < MIPS_ZERO_REGNUM + 32)
     regaddr = regno;
   else if ((regno >= mips_regnum (gdbarch)->fp0)