btrace: preserve function level for unexpected returns

When encountering a return for which we have not seen a corresponding call, GDB
starts a new back trace from level -1, i.e. from the level of the first function
in the trace.

In the presence of trace gaps, this may cause some rather big jump.

    (gdb) record function-call-history /c 192, +8
    192	                                          sbrk
    193	                                            brk
    194	                                              __x86.get_pc_thunk.bx
    195	                                            brk
    196	                                              __kernel_vsyscall
    197	[disabled]
    198	                                              __kernel_vsyscall
    199	        brk
    200	      sbrk

This doesn't help to make things more clear.  Let's remain on the same level
instead.

    (gdb) record function-call-history /c 192, +8
    192	      sbrk
    193	        brk
    194	          __x86.get_pc_thunk.bx
    195	        brk
    196	          __kernel_vsyscall
    197	[disabled]
    198	          __kernel_vsyscall
    199	        brk
    200	      sbrk

In this case it will look like we were able to connect the trace parts across
the disabled gap.  We were not.  More work is required to achieve this.

In the general case, the function-call history for the two trace parts won't
match.  They may be off by a few levels or they may be entirely different.  All
this patch does is to preserve the indentation level of the record
function-call-history command.

The disabled gap is caused by a sysenter not returning to the next instruction.

    (gdb) record function-call-history /i 196, +1
    196     __kernel_vsyscall       inst 66515,66519
    (gdb) record instruction-history 66515
    66515      0xb7fdcbf8 <__kernel_vsyscall+0>:    push   %ecx
    66516      0xb7fdcbf9 <__kernel_vsyscall+1>:    push   %edx
    66517      0xb7fdcbfa <__kernel_vsyscall+2>:    push   %ebp
    66518      0xb7fdcbfb <__kernel_vsyscall+3>:    mov    %esp,%ebp
    66519      0xb7fdcbfd <__kernel_vsyscall+5>:    sysenter
    [disabled]
    66520      0xb7fdcc08 <__kernel_vsyscall+16>:   pop    %ebp
    66521      0xb7fdcc09 <__kernel_vsyscall+17>:   pop    %edx
    66522      0xb7fdcc0a <__kernel_vsyscall+18>:   pop    %ecx
    66523      0xb7fdcc0b <__kernel_vsyscall+19>:   ret
    66524      0xb7e8e09e <brk+30>: xchg   %ecx,%ebx
    (gdb) disassemble 0xb7fdcbf8, 0xb7fdcc0c
    Dump of assembler code from 0xb7fdcbf8 to 0xb7fdcc0c:
       0xb7fdcbf8 <__kernel_vsyscall+0>:    push   %ecx
       0xb7fdcbf9 <__kernel_vsyscall+1>:    push   %edx
       0xb7fdcbfa <__kernel_vsyscall+2>:    push   %ebp
       0xb7fdcbfb <__kernel_vsyscall+3>:    mov    %esp,%ebp
       0xb7fdcbfd <__kernel_vsyscall+5>:    sysenter
       0xb7fdcbff <__kernel_vsyscall+7>:    nop
       0xb7fdcc00 <__kernel_vsyscall+8>:    nop
       0xb7fdcc01 <__kernel_vsyscall+9>:    nop
       0xb7fdcc02 <__kernel_vsyscall+10>:   nop
       0xb7fdcc03 <__kernel_vsyscall+11>:   nop
       0xb7fdcc04 <__kernel_vsyscall+12>:   nop
       0xb7fdcc05 <__kernel_vsyscall+13>:   nop
       0xb7fdcc06 <__kernel_vsyscall+14>:   int    $0x80
       0xb7fdcc08 <__kernel_vsyscall+16>:   pop    %ebp
       0xb7fdcc09 <__kernel_vsyscall+17>:   pop    %edx
       0xb7fdcc0a <__kernel_vsyscall+18>:   pop    %ecx
       0xb7fdcc0b <__kernel_vsyscall+19>:   ret
    End of assembler dump.

I've seen this on 32-bit Fedora 23.  I have not investigated what causes this
and whether we can avoid the gap in the first place.  Let's first try to make
GDB handle such gaps more gracefully.

gdb/
	* btrace.c (ftrace_new_return): Start from the previous function's level
	if we can't find a matching call for a return.

2 files changed, 18 insertions, 9 deletions

diff --git a/gdb/ChangeLog b/gdb/ChangeLog
index 98e13ea..2a601a9 100644
--- a/gdb/ChangeLog
+++ b/gdb/ChangeLog
@@ -1,5 +1,10 @@
 2016-10-28  Markus Metzger  <markus.t.metzger@intel.com>
 
+	* btrace.c (ftrace_new_return): Start from the previous function's
+	level if we can't find a matching call for a return.
+
+2016-10-28  Markus Metzger  <markus.t.metzger@intel.com>
+
 	* btrace.c (ftrace_update_function): Update tail call heuristic.
 
 2016-10-28  Markus Metzger  <markus.t.metzger@intel.com>
diff --git a/gdb/btrace.c b/gdb/btrace.c
index 445f0a4..10b6db4 100644
--- a/gdb/btrace.c
+++ b/gdb/btrace.c
@@ -387,16 +387,12 @@ ftrace_new_return (struct btrace_function *prev,
 	  /* There is no call in PREV's back trace.  We assume that the
 	     branch trace did not include it.  */
 
-	  /* Let's find the topmost call function - this skips tail calls.  */
+	  /* Let's find the topmost function and add a new caller for it.
+	     This should handle a series of initial tail calls.  */
 	  while (prev->up != NULL)
 	    prev = prev->up;
 
-	  /* We maintain levels for a series of returns for which we have
-	     not seen the calls.
-	     We start at the preceding function's level in case this has
-	     already been a return for which we have not seen the call.
-	     We start at level 0 otherwise, to handle tail calls correctly.  */
-	  bfun->level = std::min (0, prev->level) - 1;
+	  bfun->level = prev->level - 1;
 
 	  /* Fix up the call stack for PREV.  */
 	  ftrace_fixup_caller (prev, bfun, BFUN_UP_LINKS_TO_RET);
@@ -406,8 +402,16 @@ ftrace_new_return (struct btrace_function *prev,
       else
 	{
 	  /* There is a call in PREV's back trace to which we should have
-	     returned.  Let's remain at this level.  */
-	  bfun->level = prev->level;
+	     returned but didn't.  Let's start a new, separate back trace
+	     from PREV's level.  */
+	  bfun->level = prev->level - 1;
+
+	  /* We fix up the back trace for PREV but leave other function segments
+	     on the same level as they are.
+	     This should handle things like schedule () correctly where we're
+	     switching contexts.  */
+	  prev->up = bfun;
+	  prev->flags = BFUN_UP_LINKS_TO_RET;
 
 	  ftrace_debug (bfun, "new return - unknown caller");
 	}