gdbserver: Fix non-stop / fork / step-over issues

Ref: https://sourceware.org/ml/gdb-patches/2015-07/msg00868.html

This adds a test that has a multithreaded program have several threads
continuously fork, while another thread continuously steps over a
breakpoint.

This exposes several intertwined issues, which this patch addresses:

 - When we're stopping and suspending threads, some thread may fork,
   and we missed setting its suspend count to 1, like we do when a new
   clone/thread is detected.  When we next unsuspend threads, the fork
   child's suspend count goes below 0, which is bogus and fails an
   assertion.

 - If a step-over is cancelled because a signal arrives, but then gdb
   is not interested in the signal, we pass the signal straight back
   to the inferior.  However, we miss that we need to re-increment the
   suspend counts of all other threads that had been paused for the
   step-over.  As a result, other threads indefinitely end up stuck
   stopped.

 - If a detach request comes in just while gdbserver is handling a
   step-over (in the test at hand, this is GDB detaching the fork
   child), gdbserver internal errors in stabilize_thread's helpers,
   which assert that all thread's suspend counts are 0 (otherwise we
   wouldn't be able to move threads out of the jump pads).  The
   suspend counts aren't 0 while a step-over is in progress, because
   all threads but the one stepping past the breakpoint must remain
   paused until the step-over finishes and the breakpoint can be
   reinserted.

 - Occasionally, we see "BAD - reinserting but not stepping." being
   output (from within linux_resume_one_lwp_throw).  That was because
   GDB pokes memory while gdbserver is busy with a step-over, and that
   suspends threads, and then re-resumes them with proceed_one_lwp,
   which missed another reason to tell linux_resume_one_lwp that the
   thread should be set back to stepping.

 - In a couple places, we were resuming threads that are meant to be
   suspended.  E.g., when a vCont;c/s request for thread B comes in
   just while gdbserver is stepping thread A past a breakpoint.  The
   resume for thread B must be deferred until the step-over finishes.

 - The test runs with both "set detach-on-fork" on and off.  When off,
   it exercises the case of GDB detaching the fork child explicitly.
   When on, it exercises the case of gdb resuming the child
   explicitly.  In the "off" case, gdb seems to exponentially become
   slower as new inferiors are created.  This is _very_ noticeable as
   with only 100 inferiors gdb is crawling already, which makes the
   test take quite a bit to run.  For that reason, I've disabled the
   "off" variant for now.

gdb/ChangeLog:
2015-08-06  Pedro Alves  <palves@redhat.com>

	* target/waitstatus.h (enum target_stop_reason)
	<TARGET_STOPPED_BY_SINGLE_STEP>: New value.

gdb/gdbserver/ChangeLog:
2015-08-06  Pedro Alves  <palves@redhat.com>

	* linux-low.c (handle_extended_wait): Set the fork child's suspend
	count if stopping and suspending threads.
	(check_stopped_by_breakpoint): If stopped by trace, set the LWP's
	stop reason to TARGET_STOPPED_BY_SINGLE_STEP.
	(linux_detach): Complete an ongoing step-over.
	(lwp_suspended_inc, lwp_suspended_decr): New functions.  Use
	throughout.
	(resume_stopped_resumed_lwps): Don't resume a suspended thread.
	(linux_wait_1): If passing a signal to the inferior after
	finishing a step-over, unsuspend and re-resume all lwps.  If we
	see a single-step event but the thread should be continuing, don't
	pass the trap to gdb.
	(stuck_in_jump_pad_callback, move_out_of_jump_pad_callback): Use
	internal_error instead of gdb_assert.
	(enqueue_pending_signal): New function.
	(check_ptrace_stopped_lwp_gone): Add debug output.
	(start_step_over): Use internal_error instead of gdb_assert.
	(complete_ongoing_step_over): New function.
	(linux_resume_one_thread): Don't resume a suspended thread.
	(proceed_one_lwp): If the LWP is stepping over a breakpoint, reset
	it stepping.

gdb/testsuite/ChangeLog:
2015-08-06  Pedro Alves  <palves@redhat.com>

	* gdb.threads/forking-threads-plus-breakpoint.exp: New file.
	* gdb.threads/forking-threads-plus-breakpoint.c: New file.

1 files changed, 139 insertions, 0 deletions

diff --git a/gdb/testsuite/gdb.threads/forking-threads-plus-breakpoint.c b/gdb/testsuite/gdb.threads/forking-threads-plus-breakpoint.c
new file mode 100644
index 0000000..a6ff0fd
--- /dev/null
+++ b/gdb/testsuite/gdb.threads/forking-threads-plus-breakpoint.c
@@ -0,0 +1,139 @@
+/* This testcase is part of GDB, the GNU debugger.
+
+   Copyright 2015 Free Software Foundation, Inc.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+#include <assert.h>
+#include <pthread.h>
+#include <unistd.h>
+#include <stdio.h>
+#include <sys/types.h>
+#include <sys/wait.h>
+#include <stdlib.h>
+
+/* Number of threads.  Each thread continuously spawns a fork and wait
+   for it.  If we have another thread continuously start a step over,
+   gdbserver should end up finding new forks while suspending
+   threads.  */
+#define NTHREADS 10
+
+pthread_t threads[NTHREADS];
+
+pthread_barrier_t barrier;
+
+#define NFORKS 10
+
+/* Used to create a conditional breakpoint that always fails.  */
+volatile int zero;
+
+static void *
+thread_forks (void *arg)
+{
+  int i;
+
+  pthread_barrier_wait (&barrier);
+
+  for (i = 0; i < NFORKS; i++)
+    {
+      pid_t pid;
+
+      pid = fork ();
+
+      if (pid > 0)
+	{
+	  int status;
+
+	  /* Parent.  */
+	  pid = waitpid (pid, &status, 0);
+	  if (pid == -1)
+	    {
+	      perror ("wait");
+	      exit (1);
+	    }
+
+	  if (!WIFEXITED (status))
+	    {
+	      printf ("Unexpected wait status 0x%x from child %d\n",
+		      status, pid);
+	    }
+	}
+      else if (pid == 0)
+	{
+	  /* Child.  */
+	  exit (0);
+	}
+      else
+	{
+	  perror ("fork");
+	  exit (1);
+	}
+    }
+}
+
+/* Set this to tell the thread_breakpoint thread to exit.  */
+volatile int break_out;
+
+static void *
+thread_breakpoint (void *arg)
+{
+  pthread_barrier_wait (&barrier);
+
+  while (!break_out)
+    {
+      usleep (1); /* set break here */
+    }
+
+  return NULL;
+}
+
+pthread_barrier_t barrier;
+
+int
+main (void)
+{
+  int i;
+  int ret;
+  pthread_t bp_thread;
+
+  /* Don't run forever.  */
+  alarm (180);
+
+  pthread_barrier_init (&barrier, NULL, NTHREADS + 1);
+
+  /* Start the threads that constantly fork.  */
+  for (i = 0; i < NTHREADS; i++)
+    {
+      ret = pthread_create (&threads[i], NULL, thread_forks, NULL);
+      assert (ret == 0);
+    }
+
+  /* Start the thread that constantly hit a conditional breakpoint
+     that needs to be stepped over.  */
+  ret = pthread_create (&bp_thread, NULL, thread_breakpoint, NULL);
+  assert (ret == 0);
+
+  /* Wait for forking to stop.  */
+  for (i = 0; i < NTHREADS; i++)
+    {
+      ret = pthread_join (threads[i], NULL);
+      assert (ret == 0);
+    }
+
+  break_out = 1;
+  pthread_join (bp_thread, NULL);
+  assert (ret == 0);
+
+  return 0;
+}