gdb: Avoid undefined shifts, fix Go shifts

I noticed that a build of GDB with GCC + --enable-ubsan, testing
against GDBserver showed this GDB crash:

  (gdb) PASS: gdb.trace/trace-condition.exp: trace: 0x00abababcdcdcdcd << 46 == 0x7373400000000000: advance to trace begin
  tstart
  ../../src/gdb/valarith.c:1365:15: runtime error: left shift of 48320975398096333 by 46 places cannot be represented in type 'long int'
  ERROR: GDB process no longer exists
  GDB process exited with wait status 269549 exp9 0 1
  UNRESOLVED: gdb.trace/trace-condition.exp: trace: 0x00abababcdcdcdcd << 46 == 0x7373400000000000: start trace experiment

The problem is that, "0x00abababcdcdcdcd << 46" is an undefined signed
left shift, because the result is not representable in the type of the
lhs, which is signed.  This actually became defined in C++20, and if
you compile with "g++ -std=c++20 -Wall", you'll see that GCC no longer
warns about it, while it warns if you specify prior language versions.

While at it, there are a couple other situations that are undefined
(and are still undefined in C++20) and result in GDB dying: shifting
by a negative ammount, or by >= than the bit size of the promoted lhs.
For the latter, GDB shifts using (U)LONGEST internally, so you have to
shift by >= 64 bits to see it:

 $ gdb --batch -q -ex "p 1 << -1"
 ../../src/gdb/valarith.c:1365:15: runtime error: shift exponent -1 is negative
 $ # gdb exited

 $ gdb --batch -q -ex "p 1 << 64"
 ../../src/gdb/valarith.c:1365:15: runtime error: shift exponent 64 is too large for 64-bit type 'long int'
 $ # gdb exited

Also, right shifting a negative value is implementation-defined
(before C++20, after which it is defined).  For this, I chose to
change nothing in GDB other than adding tests, as I don't really know
whether we need to do anything.  AFAIK, most implementations do an
arithmetic right shift, and it may be we don't support any host or
target that behaves differently.  Plus, this becomes defined in C++20
exactly as arithmetic right shift.

Compilers don't error out on such shifts, at best they warn, so I
think GDB should just continue doing the shifts anyhow too.

Thus:

- Adjust scalar_binop to avoid the undefined paths, either by adding
  explicit result paths, or by casting the lhs of the left shift to
  unsigned, as appropriate.

  For the shifts by a too-large count, I made the result be what you'd
  get if you split the large count in a series of smaller shifts.
  Thus:

     Left shift, positive or negative lhs:

       V << 64
	 =>  V << 16 << 16 << 16 << 16
	   => 0

     Right shift, positive lhs:

       Vpos >> 64
	 =>  Vpos >> 16 >> 16 >> 16 >> 16
	   => 0

     Right shift, negative lhs:

       Vneg >> 64
	 =>  Vneg >> 16 >> 16 >> 16 >> 16
	   => -1

  This is actually Go's semantics (the compiler really emits
  instructions to make it so that you get 0 or -1 if you have a
  too-large shift).  So for that language GDB does the shift and
  nothing else.  For other C-like languages where such a shift is
  undefined, GDB warns in addition to performing the shift.

  For shift by a negative count, for Go, this is a hard error.  For
  other languages, since their compilers only warn, I made GDB warn
  too.  The semantics I chose (we're free to pick them since this is
  undefined behavior) is as-if you had shifted by the count cast to
  unsigned, thus as if you had shifted by a too-large count, thus the
  same as the previous scenario illustrated above.

  Examples:

    (gdb) set language go
    (gdb) p 1 << 100
    $1 = 0
    (gdb) p -1 << 100
    $2 = 0
    (gdb) p 1 >> 100
    $3 = 0
    (gdb) p -1 >> 100
    $4 = -1
    (gdb) p -2 >> 100
    $5 = -1
    (gdb) p 1 << -1
    left shift count is negative

    (gdb) set language c
    (gdb) p -2 >> 100
    warning: right shift count >= width of type
    $6 = -1
    (gdb) p -2 << 100
    warning: left shift count >= width of type
    $7 = 0
    (gdb) p 1 << -1
    warning: left shift count is negative
    $8 = 0
    (gdb) p -1 >> -1
    warning: right shift count is negative
    $9 = -1

- The warnings' texts are the same as what GCC prints.

- Add comprehensive tests in a new gdb.base/bitshift.exp testcase, so
  that we exercise all these scenarios.

Change-Id: I8bcd5fa02de3114b7ababc03e65702d86ec8d45d

0 files changed, 0 insertions, 0 deletions