Extend JIT-reader test and fix GDB problems that exposes

The jit-reader.exp test isn't really exercising the jit-reader's
unwinder API at all.  This commit address that, and then fixes GDB
problems exposed.

- The custom JIT reader provided for the jit-reader.exp testcase
  always rejects the jitted function's frame...

  This is because the custom JIT reader in the testcase never ever
  sets state->code_begin/end, so the bounds check in
  gdb.base/jitreader.c:unwind_frame:

   if (this_ip >= state->code_end || this_ip < state->code_begin)
     return GDB_FAIL;

  tends to fail, unless you're "lucky" (because it references
  uninitialized data).

  The result is that GDB is always actually using a built-in unwinder
  for the jitted function.

- The provided unwinder doesn't do anything that GDB's built-in
  unwinder can't do.

  IOW, we can't really tell whether the JIT reader's unwinder is
  working or not.

  I fixed that by making the jitted function mangle its own stack
  pointer with a xor, and then teaching the jit unwinder to demangle
  it back (another xor).  So now "backtrace" with GDB's built-in
  unwinder fails while with the jit unwinder, it succeeds.

- GDB crashes after unloading the JIT reader, and flushing frames...

  I made the testcase use the "flushregs" command after unloading the
  JIT reader, to force the JIT frames to be flushed.  However, that
  crashes GDB...

  When reinit_frame_cache tears down a frame's cache, it calls its
  unwinder's dealloc_cache method, which for JIT frames ends up in
  jit.c:jit_dealloc_cache.  This function calls each of the frame's
  gdb_reg_value's "free" pointer:

   for (i = 0; i < gdbarch_num_regs (frame_arch); i++)
     if (priv_data->registers[i] && priv_data->registers[i]->free)
       priv_data->registers[i]->free (priv_data->registers[i]);

  and the problem is these gdb_reg_value instances have been returned
  by the JIT reader that has been already unloaded, and their "free"
  function pointers likely point to functions in the DSO that has
  already been unloaded...

  A fix for that could be to call reinit_frame_cache in
  jit_reader_unload_command _before_ unloading the jit reader DSO so
  that the jit reader is given a chance to clean up the gdb_reg_values
  before it is unloaded.  However, the fix for the point below makes
  this unnecessary, because it stops jit.c from keeping around
  gdb_reg_values in the first place.

- However, it still makes sense to clear the frame cache when loading
  or unloading a JIT unwinder.

  This makes testing a JIT unwinder a bit simpler.

- Not only the frame cache actually -- gdb is not unloading the
  jit-registered objfiles when the JIT reader is unloaded, and not
  loading the already-registered descriptors when a JIT reader is
  loaded.

  The new test exercises unloading the jit reader, loading it back
  again, and then making sure the JIT reader's unwinder works again.
  Without the unload/re-load of already-read descriptors, the newly
  loaded JIT would have no idea where the new function is, because
  it's stored at symbol read time.

- I added a couple "info frame" calls to the test, and that
  crashes GDB...

  The problem is that jit_frame_prev_register assumes it'll only be
  called for raw registers, so when it gets a pseudo register number,
  the "priv->registers[reg]" access is really an out-of-bounds access.

  To fix that, I made jit_frame_prev_register use
  gdbarch_pseudo_register_read_value for reading the pseudo-registers.
  However, that works with a regcache and we don't have one.  To fix
  that, I made the JIT unwinder store a regcache in its cache instead
  of an array of gdb_reg_value pointers.

gdb/ChangeLog:
2016-07-01  Pedro Alves  <palves@redhat.com>
	    Tom Tromey  <tom@tromey.com>

	* jit.c (jit_reader_load_command): Call reinit_frame_cache and
	jit_inferior_created_hook.
	(jit_reader_unload_command): Call reinit_frame_cache and
	jit_inferior_exit_hook.
	* jit.c (struct jit_unwind_private) <registers>: Delete field.
	<regcache>: New field.
	(jit_unwind_reg_set_impl): Set the register's value in the
	regcache.  Free the passed-in gdb_reg_value.
	(jit_dealloc_cache): Adjust to free the regcache.
	(jit_frame_sniffer): Allocate a regcache instead of an array of
	gdb_reg_value pointers.
	(jit_frame_this_id): Adjust.
	(jit_frame_prev_register): Read raw registers off of the regcache
	instead of from the gdb_reg_value pointer array.  Use
	gdbarch_pseudo_register_read_value to read pseudo registers.
	* regcache.c (regcache_raw_set_cached_value): New function,
	factored out from ...
	(regcache_raw_write): ... here.
	* regcache.h (regcache_raw_set_cached_value): Declare.

gdb/testsuite/ChangeLog:
2016-07-01  Pedro Alves  <palves@redhat.com>

	* gdb.base/jit-reader.exp (info_registers_current_frame): New
	procedure.
	(jit_reader_test): Test the jit reader's unwinder.
	* gdb.base/jithost.c (jit_function_00_code): New global.
	(main): Use memcpy to fill in the mmapped code, instead of poking
	bytes manually here.
	* gdb.base/jitreader.c (enum register_mapping) <AMD64_RBP>: New
	value.
	(read_debug_info): Save the function's range.
	(read_sp): New function.
	(unwind_frame): Use it.  Also unwind RBP.
	(get_frame_id): Use read_sp.
	(gdb_init_reader): Use calloc instead of malloc.
	* lib/gdb.exp (get_hexadecimal_valueof): Add optional 'test'
	parameter.  Use gdb_test_multiple.

5 files changed, 272 insertions, 20 deletions

diff --git a/gdb/testsuite/ChangeLog b/gdb/testsuite/ChangeLog
index fd94379..bcbffaa 100644
--- a/gdb/testsuite/ChangeLog
+++ b/gdb/testsuite/ChangeLog
@@ -1,4 +1,22 @@
 2016-07-01  Pedro Alves  <palves@redhat.com>
+
+	* gdb.base/jit-reader.exp (info_registers_current_frame): New
+	procedure.
+	(jit_reader_test): Test the jit reader's unwinder.
+	* gdb.base/jithost.c (jit_function_00_code): New global.
+	(main): Use memcpy to fill in the mmapped code, instead of poking
+	bytes manually here.
+	* gdb.base/jitreader.c (enum register_mapping) <AMD64_RBP>: New
+	value.
+	(read_debug_info): Save the function's range.
+	(read_sp): New function.
+	(unwind_frame): Use it.  Also unwind RBP.
+	(get_frame_id): Use read_sp.
+	(gdb_init_reader): Use calloc instead of malloc.
+	* lib/gdb.exp (get_hexadecimal_valueof): Add optional 'test'
+	parameter.  Use gdb_test_multiple.
+
+2016-07-01  Pedro Alves  <palves@redhat.com>
 	    Antoine Tremblay  <antoine.tremblay@ericsson.com>
 
 	* gdb.threads/process-dies-while-detaching.c: New file.
diff --git a/gdb/testsuite/gdb.base/jit-reader.exp b/gdb/testsuite/gdb.base/jit-reader.exp
index b5e297a..642c257 100644
--- a/gdb/testsuite/gdb.base/jit-reader.exp
+++ b/gdb/testsuite/gdb.base/jit-reader.exp
@@ -57,10 +57,50 @@ if { [gdb_compile_shlib "${srcdir}/${subdir}/${jit_reader_src}" "${jit_reader_bi
     return -1
 }
 
+# Test "info registers" in the current frame, expecting RSP's value to
+# be SP.
+
+proc info_registers_current_frame {sp} {
+    global hex decimal
+
+    set any "\[^\r\n\]*"
+
+    gdb_test "info registers" \
+	[multi_line \
+	     "rax            $hex	$decimal" \
+	     "rbx            $hex	$decimal" \
+	     "rcx            $hex	$decimal" \
+	     "rdx            $hex	$decimal" \
+	     "rsi            $hex	$decimal" \
+	     "rdi            $hex	$decimal" \
+	     "rbp            $hex	$hex" \
+	     "rsp            $sp	$sp" \
+	     "r8             $hex	$decimal" \
+	     "r9             $hex	$decimal" \
+	     "r10            $hex	$decimal" \
+	     "r11            $hex	$decimal" \
+	     "r12            $hex	$decimal" \
+	     "r13            $hex	$decimal" \
+	     "r14            $hex	$decimal" \
+	     "r15            $hex	$decimal" \
+	     "rip            $hex	$hex$any" \
+	     "eflags         $hex	\\\[$any\\\]" \
+	     "cs             $hex	$decimal" \
+	     "ss             $hex	$decimal" \
+	     "ds             $hex	$decimal" \
+	     "es             $hex	$decimal" \
+	     "fs             $hex	$decimal" \
+	     "gs             $hex	$decimal" \
+	    ]
+}
+
 proc jit_reader_test {} {
     global jit_host_bin
     global jit_reader_bin
     global verbose
+    global hex decimal
+
+    set any "\[^\r\n\]*"
 
     clean_restart $jit_host_bin
     gdb_load_shlib $jit_reader_bin
@@ -73,7 +113,139 @@ proc jit_reader_test {} {
     gdb_run_cmd
     gdb_test "" "Program received signal SIGTRAP, .*" "expect SIGTRAP"
 
-    gdb_test "bt" "jit_function_00.*"
+    # Test the JIT reader unwinder.
+    with_test_prefix "with jit-reader" {
+
+	with_test_prefix "before mangling" {
+	    gdb_test "bt" \
+		[multi_line \
+		     "#0 ${any} in jit_function_00 ${any}" \
+		     "#1 ${any} in main ${any}" \
+		    ] \
+		"bt works"
+
+	    set sp_before_mangling \
+		[get_hexadecimal_valueof "\$sp" 0 "get sp"]
+
+	    gdb_test "up" "#1  $any in main $any\r\n$any  function $any" \
+		"move up to caller"
+
+	    set caller_sp \
+		[get_hexadecimal_valueof "\$sp" 0 "get caller sp"]
+	}
+
+	# Step over the instruction that mangles the stack pointer.
+	# While that confuses GDB's built-in unwinder, the JIT
+	# reader's unwinder understands the mangling and should thus
+	# be able to unwind at that location.
+	with_test_prefix "after mangling" {
+	    gdb_test "si" "in jit_function_00 .*" "step over stack mangling"
+
+	    set sp_after_mangling \
+		[get_hexadecimal_valueof "\$sp" 0 "get sp"]
+
+	    gdb_assert {$sp_before_mangling != $sp_after_mangling} \
+		"sp is mangled"
+
+	    # Check that the jit unwinder manages to backtrace through
+	    # the mangled stack pointer.
+	    gdb_test "bt" \
+		[multi_line \
+		     "#0 ${any} in jit_function_00 ${any}" \
+		     "#1 ${any} in main ${any}" \
+		    ] \
+		"bt works"
+
+	    with_test_prefix "current frame" {
+		info_registers_current_frame $sp_after_mangling
+
+		gdb_test "info frame" \
+		    "Stack level 0, frame at $sp_before_mangling.*in jit_function_00.*"
+	    }
+
+	    with_test_prefix "caller frame" {
+		gdb_test "up" "#1  $any in main $any\r\n$any  function $any" \
+		    "up to caller"
+
+		# Since the JIT unwinder only provides RIP/RSP/RBP,
+		# all other registers should show as "<not saved>".
+		gdb_test "info registers" \
+		    [multi_line \
+			 "rax            <not saved>" \
+			 "rbx            <not saved>" \
+			 "rcx            <not saved>" \
+			 "rdx            <not saved>" \
+			 "rsi            <not saved>" \
+			 "rdi            <not saved>" \
+			 "rbp            $hex	$hex" \
+			 "rsp            $caller_sp	$caller_sp" \
+			 "r8             <not saved>" \
+			 "r9             <not saved>" \
+			 "r10            <not saved>" \
+			 "r11            <not saved>" \
+			 "r12            <not saved>" \
+			 "r13            <not saved>" \
+			 "r14            <not saved>" \
+			 "r15            <not saved>" \
+			 "rip            $hex	$hex $any" \
+			 "eflags         <not saved>" \
+			 "cs             <not saved>" \
+			 "ss             <not saved>" \
+			 "ds             <not saved>" \
+			 "es             <not saved>" \
+			 "fs             <not saved>" \
+			 "gs             <not saved>" \
+			]
+
+		# Make sure that "info frame" doesn't crash.
+		gdb_test "info frame" "Stack level 1, .*in main.*"
+
+		# ... and that neither does printing a pseudo
+		# register.
+		gdb_test "print /x \$ebp" " = $hex" "print pseudo register"
+
+		# There's no way for the JIT reader API to support
+		# modifyiable values.
+		gdb_test "print \$rbp = -1" \
+		    "Attempt to assign to an unmodifiable value\." \
+		    "cannot assign to register"
+	    }
+	}
+    }
+
+    # Now unload the jit reader, and ensure that backtracing really
+    # doesn't work without it.
+    with_test_prefix "without jit-reader" {
+	gdb_test_no_output "jit-reader-unload ${jit_reader_bin}" \
+	    "jit-reader-unload"
+
+	# Check that we're no longer using the JIT unwinder, and that
+	# the built-in unwinder cannot backtrace through the mangled
+	# stack pointer.
+	gdb_test "bt" \
+	    [multi_line \
+		 "Backtrace stopped: Cannot access memory at address $sp_after_mangling" \
+		] \
+	    "bt shows error"
+
+	gdb_test "info frame" "Cannot access memory at address.*" \
+	    "info frame shows error"
+	info_registers_current_frame $sp_after_mangling
+	gdb_test "up" "Initial frame selected; you cannot go up\\." \
+	    "cannot go up"
+    }
+
+    with_test_prefix "with jit-reader again" {
+	gdb_test_no_output "jit-reader-load ${jit_reader_bin}" "jit-reader-load"
+
+	# Check that the jit unwinder manages to backtrace through
+	# the mangled stack pointer.
+	gdb_test "bt" \
+	    [multi_line \
+		 "#0 ${any} in jit_function_00 ${any}" \
+		 "#1 ${any} in main ${any}" \
+		]
+    }
 }
 
 jit_reader_test
diff --git a/gdb/testsuite/gdb.base/jithost.c b/gdb/testsuite/gdb.base/jithost.c
index 09ab377..afefc98 100644
--- a/gdb/testsuite/gdb.base/jithost.c
+++ b/gdb/testsuite/gdb.base/jithost.c
@@ -33,18 +33,32 @@ struct jit_code_entry only_entry;
 
 typedef void (jit_function_t) ();
 
-int main (int argc, char **argv)
+/* The code of the jit_function_00 function that is copied into an
+   mmapped buffer in the inferior at run time.
+
+   The second instruction mangles the stack pointer, meaning that when
+   stopped at the third instruction, GDB needs assistance from the JIT
+   unwinder in order to be able to unwind successfully.  */
+const unsigned char jit_function_00_code[] = {
+  0xcc,				/* int3 */
+  0x48, 0x83, 0xf4, 0xff,	/* xor $0xffffffffffffffff, %rsp */
+  0x48, 0x83, 0xf4, 0xff,	/* xor $0xffffffffffffffff, %rsp */
+  0xc3				/* ret */
+};
+
+int
+main (int argc, char **argv)
 {
+  struct jithost_abi *symfile;
   char *code = mmap (NULL, getpagesize (), PROT_WRITE | PROT_EXEC,
 		     MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
   jit_function_t *function = (jit_function_t *) code;
 
-  code[0] = 0xcc; /* SIGTRAP  */
-  code[1] = 0xc3; /* RET  */
+  memcpy (code, jit_function_00_code, sizeof (jit_function_00_code));
 
-  struct jithost_abi *symfile = malloc (sizeof (struct jithost_abi));
+  symfile = malloc (sizeof (struct jithost_abi));
   symfile->begin = code;
-  symfile->end = code + 2;
+  symfile->end = code + sizeof (jit_function_00_code);
 
   only_entry.symfile_addr = symfile;
   only_entry.symfile_size = sizeof (struct jithost_abi);
diff --git a/gdb/testsuite/gdb.base/jitreader.c b/gdb/testsuite/gdb.base/jitreader.c
index 850fb46..c18e40d 100644
--- a/gdb/testsuite/gdb.base/jitreader.c
+++ b/gdb/testsuite/gdb.base/jitreader.c
@@ -28,6 +28,7 @@ GDB_DECLARE_GPL_COMPATIBLE_READER;
 enum register_mapping
 {
   AMD64_RA = 16,
+  AMD64_RBP = 6,
   AMD64_RSP = 7,
 };
 
@@ -47,6 +48,11 @@ read_debug_info (struct gdb_reader_funcs *self,
   struct gdb_symtab *symtab = cbs->symtab_open (cbs, object, "");
   GDB_CORE_ADDR begin = (GDB_CORE_ADDR) symfile->begin;
   GDB_CORE_ADDR end = (GDB_CORE_ADDR) symfile->end;
+  struct reader_state *state = (struct reader_state *) self->priv_data;
+
+  /* Record the function's range, for the unwinder.  */
+  state->code_begin = begin;
+  state->code_end = end;
 
   cbs->block_open (cbs, symtab, NULL, begin, end, "jit_function_00");
 
@@ -91,11 +97,36 @@ read_register (struct gdb_unwind_callbacks *callbacks, int dw_reg,
   return 1;
 }
 
+/* Read the stack pointer into *VALUE.  IP is the address the inferior
+   is currently stopped at.  Takes care of demangling the stack
+   pointer if necessary.  */
+
+static int
+read_sp (struct gdb_reader_funcs *self, struct gdb_unwind_callbacks *cbs,
+	 uintptr_t ip, uintptr_t *value)
+{
+  struct reader_state *state = (struct reader_state *) self->priv_data;
+  uintptr_t sp;
+
+  if (!read_register (cbs, AMD64_RSP, &sp))
+    return GDB_FAIL;
+
+  /* If stopped at the instruction after the "xor $-1, %rsp", demangle
+     the stack pointer back.  */
+  if (ip == state->code_begin + 5)
+    sp ^= (uintptr_t) -1;
+
+  *value = sp;
+  return GDB_SUCCESS;
+}
+
 static enum gdb_status
 unwind_frame (struct gdb_reader_funcs *self, struct gdb_unwind_callbacks *cbs)
 {
   const int word_size = sizeof (uintptr_t);
-  uintptr_t this_sp, this_ip, prev_ip, prev_sp;
+  uintptr_t prev_sp, this_sp;
+  uintptr_t prev_ip, this_ip;
+  uintptr_t prev_bp, this_bp;
   struct reader_state *state = (struct reader_state *) self->priv_data;
 
   if (!read_register (cbs, AMD64_RA, &this_ip))
@@ -104,15 +135,25 @@ unwind_frame (struct gdb_reader_funcs *self, struct gdb_unwind_callbacks *cbs)
   if (this_ip >= state->code_end || this_ip < state->code_begin)
     return GDB_FAIL;
 
-  if (!read_register (cbs, AMD64_RSP, &this_sp))
+  /* Unwind RBP in order to make the unwinder that tries to unwind
+     from the just-unwound frame happy.  */
+  if (!read_register (cbs, AMD64_RBP, &this_bp))
     return GDB_FAIL;
+  /* RBP is unmodified.  */
+  prev_bp = this_bp;
 
-  if (cbs->target_read (this_sp, &prev_ip, word_size) == GDB_FAIL)
+  /* Fetch the demangled stack pointer.  */
+  if (!read_sp (self, cbs, this_ip, &this_sp))
     return GDB_FAIL;
 
+  /* The return address is saved on the stack.  */
+  if (cbs->target_read (this_sp, &prev_ip, word_size) == GDB_FAIL)
+    return GDB_FAIL;
   prev_sp = this_sp + word_size;
+
   write_register (cbs, AMD64_RA, prev_ip);
   write_register (cbs, AMD64_RSP, prev_sp);
+  write_register (cbs, AMD64_RBP, prev_bp);
   return GDB_SUCCESS;
 }
 
@@ -121,9 +162,11 @@ get_frame_id (struct gdb_reader_funcs *self, struct gdb_unwind_callbacks *cbs)
 {
   struct reader_state *state = (struct reader_state *) self->priv_data;
   struct gdb_frame_id frame_id;
-
+  uintptr_t ip;
   uintptr_t sp;
-  read_register (cbs, AMD64_RSP, &sp);
+
+  read_register (cbs, AMD64_RA, &ip);
+  read_sp (self, cbs, ip, &sp);
 
   frame_id.code_address = (GDB_CORE_ADDR) state->code_begin;
   frame_id.stack_address = (GDB_CORE_ADDR) sp;
@@ -141,7 +184,7 @@ destroy_reader (struct gdb_reader_funcs *self)
 struct gdb_reader_funcs *
 gdb_init_reader (void)
 {
-  struct reader_state *state = malloc (sizeof (struct reader_state));
+  struct reader_state *state = calloc (1, sizeof (struct reader_state));
   struct gdb_reader_funcs *reader_funcs =
     malloc (sizeof (struct gdb_reader_funcs));
 
diff --git a/gdb/testsuite/lib/gdb.exp b/gdb/testsuite/lib/gdb.exp
index b52fdd9..b7b8fad 100644
--- a/gdb/testsuite/lib/gdb.exp
+++ b/gdb/testsuite/lib/gdb.exp
@@ -5435,19 +5435,24 @@ proc get_integer_valueof { exp default } {
     return ${val}
 }
 
-proc get_hexadecimal_valueof { exp default } {
+# Retrieve the value of EXP in the inferior, as an hexadecimal value
+# (using "print /x").  DEFAULT is used as fallback if print fails.
+# TEST is the test message to use.  If can be ommitted, in which case
+# a test message is built from EXP.
+
+proc get_hexadecimal_valueof { exp default {test ""} } {
     global gdb_prompt
-    send_gdb "print /x ${exp}\n"
-    set test "get hexadecimal valueof \"${exp}\""
-    gdb_expect {
+
+    if {$test == ""} {
+	set test "get hexadecimal valueof \"${exp}\""
+    }
+
+    set val ${default}
+    gdb_test_multiple "print /x ${exp}" $test {
 	-re "\\$\[0-9\]* = (0x\[0-9a-zA-Z\]+).*$gdb_prompt $" {
 	    set val $expect_out(1,string)
 	    pass "$test"
 	}
-	timeout {
-	    set val ${default}
-	    fail "$test (timeout)"
-	}
     }
     return ${val}
 }