Linux: on attach, attach to lwps listed under /proc/$pid/task/

... instead of relying on libthread_db.

I wrote a test that attaches to a program that constantly spawns
short-lived threads, which exposed several issues.  This is one of
them.

On Linux, we need to attach to all threads of a process (thread group)
individually.  We currently rely on libthread_db to list the threads,
but that is problematic, because libthread_db relies on reading data
structures out of the inferior (which may well be corrupted).  If
threads are being created or exiting just while we try to attach, we
may trip on inconsistencies in the inferior's thread list.  To work
around that, when we see a seemingly corrupt list, we currently retry
a few times:

 static void
 thread_db_find_new_threads_2 (ptid_t ptid, int until_no_new)
 {
 ...
   if (until_no_new)
     {
       /* Require 4 successive iterations which do not find any new threads.
	  The 4 is a heuristic: there is an inherent race here, and I have
	  seen that 2 iterations in a row are not always sufficient to
	  "capture" all threads.  */
 ...

That heuristic may well fail, and when it does, we end up with threads
in the program that aren't under GDB's control.  That's obviously bad
and results in quite mistifying failures, like e.g., the process dying
for seeminly no reason when a thread that wasn't attached trips on a
breakpoint.

There's really no reason to rely on libthread_db for this nowadays
when we have /proc mounted.  In that case, which is the usual case, we
can list the LWPs from /proc/PID/task/.  In fact, GDBserver is already
doing this.  The patch factors out that code that knows to walk the
task/ directory out of GDBserver, and makes GDB use it too.

Like GDBserver, the patch makes GDB attach to LWPs and _not_ wait for
them to stop immediately.  Instead, we just tag the LWP as having an
expected stop.  Because we can only set the ptrace options when the
thread stops, we need a new flag in the lwp structure to keep track of
whether we've already set the ptrace options, just like in GDBserver.
Note that nothing issues any ptrace command to the threads between the
PTRACE_ATTACH and the stop, so this is safe (unlike one scenario
described in gdbserver's linux-low.c).

When we attach to a program that has threads exiting while we attach,
it's easy to race with a thread just exiting as we try to attach to
it, like:

  #1 - get current list of threads
  #2 - attach to each listed thread
  #3 - ooops, attach failed, thread is already gone

As this is pretty normal, we shouldn't be issuing a scary warning in
step #3.

When #3 happens, PTRACE_ATTACH usually fails with ESRCH, but sometimes
we'll see EPERM as well.  That happens when the kernel still has the
thread in its task list, but the thread is marked as dead.
Unfortunately, EPERM is ambiguous and we'll get it also on other
scenarios where the thread isn't dead, and in those cases, it's useful
to get a warning.  To distiguish the cases, when we get an EPERM
failure, we open /proc/PID/status, and check the thread's state -- if
the /proc file no longer exists, or the state is "Z (Zombie)" or "X
(Dead)", we ignore the EPERM error silently; otherwise, we'll warn.
Unfortunately, there seems to be a kernel race here.  Sometimes I get
EPERM, and then the /proc state still indicates "R (Running)"...  If
we wait a bit and retry, we do end up seeing X or Z state, or get an
ESRCH.  I thought of making GDB retry the attach a few times, but even
with a 500ms wait and 4 retries, I still see the warning sometimes.  I
haven't been able to identify the kernel path that causes this yet,
but in any case, it looks like a kernel bug to me.  As this just
results failure to suppress a warning that we've been printing since
about forever anyway, I'm just making the test cope with it, and issue
an XFAIL.

gdb/gdbserver/
2015-01-09  Pedro Alves  <palves@redhat.com>

	* linux-low.c (linux_attach_fail_reason_string): Move to
	nat/linux-ptrace.c, and rename.
	(linux_attach_lwp): Update comment.
	(attach_proc_task_lwp_callback): New function.
	(linux_attach): Adjust to rename and use
	linux_proc_attach_tgid_threads.
	(linux_attach_fail_reason_string): Delete declaration.

gdb/
2015-01-09  Pedro Alves  <palves@redhat.com>

	* linux-nat.c (attach_proc_task_lwp_callback): New function.
	(linux_nat_attach): Use linux_proc_attach_tgid_threads.
	(wait_lwp, linux_nat_filter_event): If not set yet, set the lwp's
	ptrace option flags.
	* linux-nat.h (struct lwp_info) <must_set_ptrace_flags>: New
	field.
	* nat/linux-procfs.c: Include <dirent.h>.
	(linux_proc_get_int): New parameter "warn".  Handle it.
	(linux_proc_get_tgid): Adjust.
	(linux_proc_get_tracerpid): Rename to ...
	(linux_proc_get_tracerpid_nowarn): ... this.
	(linux_proc_pid_get_state): New function, factored out from
	(linux_proc_pid_has_state): ... this.  Add new parameter "warn"
	and handle it.
	(linux_proc_pid_is_gone): New function.
	(linux_proc_pid_is_stopped): Adjust.
	(linux_proc_pid_is_zombie_maybe_warn)
	(linux_proc_pid_is_zombie_nowarn): New functions.
	(linux_proc_pid_is_zombie): Use
	linux_proc_pid_is_zombie_maybe_warn.
	(linux_proc_attach_tgid_threads): New function.
	* nat/linux-procfs.h (linux_proc_get_tgid): Update comment.
	(linux_proc_get_tracerpid): Rename to ...
	(linux_proc_get_tracerpid_nowarn): ... this, and update comment.
	(linux_proc_pid_is_gone): New declaration.
	(linux_proc_pid_is_zombie): Update comment.
	(linux_proc_pid_is_zombie_nowarn): New declaration.
	(linux_proc_attach_lwp_func): New typedef.
	(linux_proc_attach_tgid_threads): New declaration.
	* nat/linux-ptrace.c (linux_ptrace_attach_fail_reason): Adjust to
	use nowarn functions.
	(linux_ptrace_attach_fail_reason_string): Move here from
	gdbserver/linux-low.c and rename.
	(ptrace_supports_feature): If the current ptrace options are not
	known yet, check them now, instead of asserting.
	* nat/linux-ptrace.h (linux_ptrace_attach_fail_reason_string):
	Declare.

11 files changed, 406 insertions, 127 deletions

diff --git a/gdb/ChangeLog b/gdb/ChangeLog
index 6eb4926..4ee17b3 100644
--- a/gdb/ChangeLog
+++ b/gdb/ChangeLog
@@ -1,5 +1,45 @@
 2015-01-09  Pedro Alves  <palves@redhat.com>
 
+	* linux-nat.c (attach_proc_task_lwp_callback): New function.
+	(linux_nat_attach): Use linux_proc_attach_tgid_threads.
+	(wait_lwp, linux_nat_filter_event): If not set yet, set the lwp's
+	ptrace option flags.
+	* linux-nat.h (struct lwp_info) <must_set_ptrace_flags>: New
+	field.
+	* nat/linux-procfs.c: Include <dirent.h>.
+	(linux_proc_get_int): New parameter "warn".  Handle it.
+	(linux_proc_get_tgid): Adjust.
+	(linux_proc_get_tracerpid): Rename to ...
+	(linux_proc_get_tracerpid_nowarn): ... this.
+	(linux_proc_pid_get_state): New function, factored out from
+	(linux_proc_pid_has_state): ... this.  Add new parameter "warn"
+	and handle it.
+	(linux_proc_pid_is_gone): New function.
+	(linux_proc_pid_is_stopped): Adjust.
+	(linux_proc_pid_is_zombie_maybe_warn)
+	(linux_proc_pid_is_zombie_nowarn): New functions.
+	(linux_proc_pid_is_zombie): Use
+	linux_proc_pid_is_zombie_maybe_warn.
+	(linux_proc_attach_tgid_threads): New function.
+	* nat/linux-procfs.h (linux_proc_get_tgid): Update comment.
+	(linux_proc_get_tracerpid): Rename to ...
+	(linux_proc_get_tracerpid_nowarn): ... this, and update comment.
+	(linux_proc_pid_is_gone): New declaration.
+	(linux_proc_pid_is_zombie): Update comment.
+	(linux_proc_pid_is_zombie_nowarn): New declaration.
+	(linux_proc_attach_lwp_func): New typedef.
+	(linux_proc_attach_tgid_threads): New declaration.
+	* nat/linux-ptrace.c (linux_ptrace_attach_fail_reason): Adjust to
+	use nowarn functions.
+	(linux_ptrace_attach_fail_reason_string): Move here from
+	gdbserver/linux-low.c and rename.
+	(ptrace_supports_feature): If the current ptrace options are not
+	known yet, check them now, instead of asserting.
+	* nat/linux-ptrace.h (linux_ptrace_attach_fail_reason_string):
+	Declare.
+
+2015-01-09  Pedro Alves  <palves@redhat.com>
+
 	* linux-thread-db.c (thread_db_find_new_threads_silently)
 	(try_thread_db_load_1, try_thread_db_load, thread_db_load_search)
 	(find_new_threads_once): Print debug output on gdb_stdlog.
diff --git a/gdb/gdbserver/ChangeLog b/gdb/gdbserver/ChangeLog
index 6937ec0..2ee6a00 100644
--- a/gdb/gdbserver/ChangeLog
+++ b/gdb/gdbserver/ChangeLog
@@ -1,3 +1,13 @@
+2015-01-09  Pedro Alves  <palves@redhat.com>
+
+	* linux-low.c (linux_attach_fail_reason_string): Move to
+	nat/linux-ptrace.c, and rename.
+	(linux_attach_lwp): Update comment.
+	(attach_proc_task_lwp_callback): New function.
+	(linux_attach): Adjust to rename and use
+	linux_proc_attach_tgid_threads.
+	(linux_attach_fail_reason_string): Delete declaration.
+
 2015-01-01  Joel Brobecker  <brobecker@adacore.com>
 
 	* gdbreplay.c (gdbreplay_version): Update copyright year to 2015.
diff --git a/gdb/gdbserver/linux-low.c b/gdb/gdbserver/linux-low.c
index 0d85189..268ee5c 100644
--- a/gdb/gdbserver/linux-low.c
+++ b/gdb/gdbserver/linux-low.c
@@ -631,31 +631,8 @@ linux_create_inferior (char *program, char **allargs)
   return pid;
 }
 
-char *
-linux_attach_fail_reason_string (ptid_t ptid, int err)
-{
-  static char *reason_string;
-  struct buffer buffer;
-  char *warnings;
-  long lwpid = ptid_get_lwp (ptid);
-
-  xfree (reason_string);
-
-  buffer_init (&buffer);
-  linux_ptrace_attach_fail_reason (lwpid, &buffer);
-  buffer_grow_str0 (&buffer, "");
-  warnings = buffer_finish (&buffer);
-  if (warnings[0] != '\0')
-    reason_string = xstrprintf ("%s (%d), %s",
-				strerror (err), err, warnings);
-  else
-    reason_string = xstrprintf ("%s (%d)",
-				strerror (err), err);
-  xfree (warnings);
-  return reason_string;
-}
-
-/* Attach to an inferior process.  */
+/* Attach to an inferior process.  Returns 0 on success, ERRNO on
+   error.  */
 
 int
 linux_attach_lwp (ptid_t ptid)
@@ -739,6 +716,50 @@ linux_attach_lwp (ptid_t ptid)
   return 0;
 }
 
+/* Callback for linux_proc_attach_tgid_threads.  Attach to PTID if not
+   already attached.  Returns true if a new LWP is found, false
+   otherwise.  */
+
+static int
+attach_proc_task_lwp_callback (ptid_t ptid)
+{
+  /* Is this a new thread?  */
+  if (find_thread_ptid (ptid) == NULL)
+    {
+      int lwpid = ptid_get_lwp (ptid);
+      int err;
+
+      if (debug_threads)
+	debug_printf ("Found new lwp %d\n", lwpid);
+
+      err = linux_attach_lwp (ptid);
+
+      /* Be quiet if we simply raced with the thread exiting.  EPERM
+	 is returned if the thread's task still exists, and is marked
+	 as exited or zombie, as well as other conditions, so in that
+	 case, confirm the status in /proc/PID/status.  */
+      if (err == ESRCH
+	  || (err == EPERM && linux_proc_pid_is_gone (lwpid)))
+	{
+	  if (debug_threads)
+	    {
+	      debug_printf ("Cannot attach to lwp %d: "
+			    "thread is gone (%d: %s)\n",
+			    lwpid, err, strerror (err));
+	    }
+	}
+      else if (err != 0)
+	{
+	  warning (_("Cannot attach to lwp %d: %s"),
+		   lwpid,
+		   linux_ptrace_attach_fail_reason_string (ptid, err));
+	}
+
+      return 1;
+    }
+  return 0;
+}
+
 /* Attach to PID.  If PID is the tgid, attach to it and all
    of its threads.  */
 
@@ -753,7 +774,7 @@ linux_attach (unsigned long pid)
   err = linux_attach_lwp (ptid);
   if (err != 0)
     error ("Cannot attach to process %ld: %s",
-	   pid, linux_attach_fail_reason_string (ptid, err));
+	   pid, linux_ptrace_attach_fail_reason_string (ptid, err));
 
   linux_add_process (pid, 1);
 
@@ -767,75 +788,16 @@ linux_attach (unsigned long pid)
       thread->last_resume_kind = resume_stop;
     }
 
-  if (linux_proc_get_tgid (pid) == pid)
-    {
-      DIR *dir;
-      char pathname[128];
-
-      sprintf (pathname, "/proc/%ld/task", pid);
-
-      dir = opendir (pathname);
-
-      if (!dir)
-	{
-	  fprintf (stderr, "Could not open /proc/%ld/task.\n", pid);
-	  fflush (stderr);
-	}
-      else
-	{
-	  /* At this point we attached to the tgid.  Scan the task for
-	     existing threads.  */
-	  int new_threads_found;
-	  int iterations = 0;
-
-	  while (iterations < 2)
-	    {
-	      struct dirent *dp;
-
-	      new_threads_found = 0;
-	      /* Add all the other threads.  While we go through the
-		 threads, new threads may be spawned.  Cycle through
-		 the list of threads until we have done two iterations without
-		 finding new threads.  */
-	      while ((dp = readdir (dir)) != NULL)
-		{
-		  unsigned long lwp;
-		  ptid_t ptid;
-
-		  /* Fetch one lwp.  */
-		  lwp = strtoul (dp->d_name, NULL, 10);
-
-		  ptid = ptid_build (pid, lwp, 0);
-
-		  /* Is this a new thread?  */
-		  if (lwp != 0 && find_thread_ptid (ptid) == NULL)
-		    {
-		      int err;
-
-		      if (debug_threads)
-			debug_printf ("Found new lwp %ld\n", lwp);
-
-		      err = linux_attach_lwp (ptid);
-		      if (err != 0)
-			warning ("Cannot attach to lwp %ld: %s",
-				 lwp,
-				 linux_attach_fail_reason_string (ptid, err));
-
-		      new_threads_found++;
-		    }
-		}
-
-	      if (!new_threads_found)
-		iterations++;
-	      else
-		iterations = 0;
-
-	      rewinddir (dir);
-	    }
-	  closedir (dir);
-	}
-    }
-
+  /* We must attach to every LWP.  If /proc is mounted, use that to
+     find them now.  On the one hand, the inferior may be using raw
+     clone instead of using pthreads.  On the other hand, even if it
+     is using pthreads, GDB may not be connected yet (thread_db needs
+     to do symbol lookups, through qSymbol).  Also, thread_db walks
+     structures in the inferior's address space to find the list of
+     threads/LWPs, and those structures may well be corrupted.  Note
+     that once thread_db is loaded, we'll still use it to list threads
+     and associate pthread info with each LWP.  */
+  linux_proc_attach_tgid_threads (pid, attach_proc_task_lwp_callback);
   return 0;
 }
 
diff --git a/gdb/gdbserver/linux-low.h b/gdb/gdbserver/linux-low.h
index 1a1d69c..97b163f 100644
--- a/gdb/gdbserver/linux-low.h
+++ b/gdb/gdbserver/linux-low.h
@@ -351,12 +351,6 @@ int linux_pid_exe_is_elf_64_file (int pid, unsigned int *machine);
    errno).  */
 int linux_attach_lwp (ptid_t ptid);
 
-/* Return the reason an attach failed, in string form.  ERR is the
-   error returned by linux_attach_lwp (an errno).  This string should
-   be copied into a buffer by the client if the string will not be
-   immediately used, or if it must persist.  */
-char *linux_attach_fail_reason_string (ptid_t ptid, int err);
-
 struct lwp_info *find_lwp_pid (ptid_t ptid);
 void linux_stop_lwp (struct lwp_info *lwp);
 
diff --git a/gdb/gdbserver/thread-db.c b/gdb/gdbserver/thread-db.c
index 71bc984..4e0d32a 100644
--- a/gdb/gdbserver/thread-db.c
+++ b/gdb/gdbserver/thread-db.c
@@ -339,7 +339,7 @@ attach_thread (const td_thrhandle_t *th_p, td_thrinfo_t *ti_p)
     {
       warning ("Could not attach to thread %ld (LWP %d): %s\n",
 	       ti_p->ti_tid, ti_p->ti_lid,
-	       linux_attach_fail_reason_string (ptid, err));
+	       linux_ptrace_attach_fail_reason_string (ptid, err));
       return 0;
     }
 
diff --git a/gdb/linux-nat.c b/gdb/linux-nat.c
index 73fd2cb..0adf3a9 100644
--- a/gdb/linux-nat.c
+++ b/gdb/linux-nat.c
@@ -1137,6 +1137,73 @@ linux_nat_create_inferior (struct target_ops *ops,
 #endif /* HAVE_PERSONALITY */
 }
 
+/* Callback for linux_proc_attach_tgid_threads.  Attach to PTID if not
+   already attached.  Returns true if a new LWP is found, false
+   otherwise.  */
+
+static int
+attach_proc_task_lwp_callback (ptid_t ptid)
+{
+  struct lwp_info *lp;
+
+  /* Ignore LWPs we're already attached to.  */
+  lp = find_lwp_pid (ptid);
+  if (lp == NULL)
+    {
+      int lwpid = ptid_get_lwp (ptid);
+
+      if (ptrace (PTRACE_ATTACH, lwpid, 0, 0) < 0)
+	{
+	  int err = errno;
+
+	  /* Be quiet if we simply raced with the thread exiting.
+	     EPERM is returned if the thread's task still exists, and
+	     is marked as exited or zombie, as well as other
+	     conditions, so in that case, confirm the status in
+	     /proc/PID/status.  */
+	  if (err == ESRCH
+	      || (err == EPERM && linux_proc_pid_is_gone (lwpid)))
+	    {
+	      if (debug_linux_nat)
+		{
+		  fprintf_unfiltered (gdb_stdlog,
+				      "Cannot attach to lwp %d: "
+				      "thread is gone (%d: %s)\n",
+				      lwpid, err, safe_strerror (err));
+		}
+	    }
+	  else
+	    {
+	      warning (_("Cannot attach to lwp %d: %s\n"),
+		       lwpid,
+		       linux_ptrace_attach_fail_reason_string (ptid,
+							       err));
+	    }
+	}
+      else
+	{
+	  if (debug_linux_nat)
+	    fprintf_unfiltered (gdb_stdlog,
+				"PTRACE_ATTACH %s, 0, 0 (OK)\n",
+				target_pid_to_str (ptid));
+
+	  lp = add_lwp (ptid);
+	  lp->cloned = 1;
+
+	  /* The next time we wait for this LWP we'll see a SIGSTOP as
+	     PTRACE_ATTACH brings it to a halt.  */
+	  lp->signalled = 1;
+
+	  /* We need to wait for a stop before being able to make the
+	     next ptrace call on this LWP.  */
+	  lp->must_set_ptrace_flags = 1;
+	}
+
+      return 1;
+    }
+  return 0;
+}
+
 static void
 linux_nat_attach (struct target_ops *ops, const char *args, int from_tty)
 {
@@ -1230,6 +1297,16 @@ linux_nat_attach (struct target_ops *ops, const char *args, int from_tty)
 
   lp->status = status;
 
+  /* We must attach to every LWP.  If /proc is mounted, use that to
+     find them now.  The inferior may be using raw clone instead of
+     using pthreads.  But even if it is using pthreads, thread_db
+     walks structures in the inferior's address space to find the list
+     of threads/LWPs, and those structures may well be corrupted.
+     Note that once thread_db is loaded, we'll still use it to list
+     threads and associate pthread info with each LWP.  */
+  linux_proc_attach_tgid_threads (ptid_get_pid (lp->ptid),
+				  attach_proc_task_lwp_callback);
+
   if (target_can_async_p ())
     target_async (inferior_event_handler, 0);
 }
@@ -2159,6 +2236,14 @@ wait_lwp (struct lwp_info *lp)
   gdb_assert (WIFSTOPPED (status));
   lp->stopped = 1;
 
+  if (lp->must_set_ptrace_flags)
+    {
+      struct inferior *inf = find_inferior_pid (ptid_get_pid (lp->ptid));
+
+      linux_enable_event_reporting (ptid_get_lwp (lp->ptid), inf->attach_flag);
+      lp->must_set_ptrace_flags = 0;
+    }
+
   /* Handle GNU/Linux's syscall SIGTRAPs.  */
   if (WIFSTOPPED (status) && WSTOPSIG (status) == SYSCALL_SIGTRAP)
     {
@@ -2783,6 +2868,14 @@ linux_nat_filter_event (int lwpid, int status, int *new_pending_p)
      ever being continued.)  */
   lp->stopped = 1;
 
+  if (WIFSTOPPED (status) && lp->must_set_ptrace_flags)
+    {
+      struct inferior *inf = find_inferior_pid (ptid_get_pid (lp->ptid));
+
+      linux_enable_event_reporting (ptid_get_lwp (lp->ptid), inf->attach_flag);
+      lp->must_set_ptrace_flags = 0;
+    }
+
   /* Handle GNU/Linux's syscall SIGTRAPs.  */
   if (WIFSTOPPED (status) && WSTOPSIG (status) == SYSCALL_SIGTRAP)
     {
diff --git a/gdb/linux-nat.h b/gdb/linux-nat.h
index b65b2fe..8a44324 100644
--- a/gdb/linux-nat.h
+++ b/gdb/linux-nat.h
@@ -33,6 +33,10 @@ struct lwp_info
      and overall process id.  */
   ptid_t ptid;
 
+  /* If this flag is set, we need to set the event request flags the
+     next time we see this LWP stop.  */
+  int must_set_ptrace_flags;
+
   /* Non-zero if this LWP is cloned.  In this context "cloned" means
      that the LWP is reporting to its parent using a signal other than
      SIGCHLD.  */
diff --git a/gdb/nat/linux-procfs.c b/gdb/nat/linux-procfs.c
index f020d27..00b6a70 100644
--- a/gdb/nat/linux-procfs.c
+++ b/gdb/nat/linux-procfs.c
@@ -19,12 +19,13 @@
 #include "common-defs.h"
 #include "linux-procfs.h"
 #include "filestuff.h"
+#include <dirent.h>
 
 /* Return the TGID of LWPID from /proc/pid/status.  Returns -1 if not
    found.  */
 
 static int
-linux_proc_get_int (pid_t lwpid, const char *field)
+linux_proc_get_int (pid_t lwpid, const char *field, int warn)
 {
   size_t field_len = strlen (field);
   FILE *status_file;
@@ -35,7 +36,8 @@ linux_proc_get_int (pid_t lwpid, const char *field)
   status_file = gdb_fopen_cloexec (buf, "r");
   if (status_file == NULL)
     {
-      warning (_("unable to open /proc file '%s'"), buf);
+      if (warn)
+	warning (_("unable to open /proc file '%s'"), buf);
       return -1;
     }
 
@@ -56,45 +58,87 @@ linux_proc_get_int (pid_t lwpid, const char *field)
 int
 linux_proc_get_tgid (pid_t lwpid)
 {
-  return linux_proc_get_int (lwpid, "Tgid");
+  return linux_proc_get_int (lwpid, "Tgid", 1);
 }
 
 /* See linux-procfs.h.  */
 
 pid_t
-linux_proc_get_tracerpid (pid_t lwpid)
+linux_proc_get_tracerpid_nowarn (pid_t lwpid)
 {
-  return linux_proc_get_int (lwpid, "TracerPid");
+  return linux_proc_get_int (lwpid, "TracerPid", 0);
 }
 
-/* Return non-zero if 'State' of /proc/PID/status contains STATE.  */
+/* Fill in BUFFER, a buffer with BUFFER_SIZE bytes with the 'State'
+   line of /proc/PID/status.  Returns -1 on failure to open the /proc
+   file, 1 if the line is found, and 0 if not found.  If WARN, warn on
+   failure to open the /proc file.  */
 
 static int
-linux_proc_pid_has_state (pid_t pid, const char *state)
+linux_proc_pid_get_state (pid_t pid, char *buffer, size_t buffer_size,
+			  int warn)
 {
-  char buffer[100];
   FILE *procfile;
-  int retval;
   int have_state;
 
-  xsnprintf (buffer, sizeof (buffer), "/proc/%d/status", (int) pid);
+  xsnprintf (buffer, buffer_size, "/proc/%d/status", (int) pid);
   procfile = gdb_fopen_cloexec (buffer, "r");
   if (procfile == NULL)
     {
-      warning (_("unable to open /proc file '%s'"), buffer);
-      return 0;
+      if (warn)
+	warning (_("unable to open /proc file '%s'"), buffer);
+      return -1;
     }
 
   have_state = 0;
-  while (fgets (buffer, sizeof (buffer), procfile) != NULL)
+  while (fgets (buffer, buffer_size, procfile) != NULL)
     if (strncmp (buffer, "State:", 6) == 0)
       {
 	have_state = 1;
 	break;
       }
-  retval = (have_state && strstr (buffer, state) != NULL);
   fclose (procfile);
-  return retval;
+  return have_state;
+}
+
+/* See linux-procfs.h declaration.  */
+
+int
+linux_proc_pid_is_gone (pid_t pid)
+{
+  char buffer[100];
+  int have_state;
+
+  have_state = linux_proc_pid_get_state (pid, buffer, sizeof buffer, 0);
+  if (have_state < 0)
+    {
+      /* If we can't open the status file, assume the thread has
+	 disappeared.  */
+      return 1;
+    }
+  else if (have_state == 0)
+    {
+      /* No "State:" line, assume thread is alive.  */
+      return 0;
+    }
+  else
+    {
+      return (strstr (buffer, "Z (") != NULL
+	      || strstr (buffer, "X (") != NULL);
+    }
+}
+
+/* Return non-zero if 'State' of /proc/PID/status contains STATE.  If
+   WARN, warn on failure to open the /proc file.  */
+
+static int
+linux_proc_pid_has_state (pid_t pid, const char *state, int warn)
+{
+  char buffer[100];
+  int have_state;
+
+  have_state = linux_proc_pid_get_state (pid, buffer, sizeof buffer, warn);
+  return (have_state > 0 && strstr (buffer, state) != NULL);
 }
 
 /* Detect `T (stopped)' in `/proc/PID/status'.
@@ -103,7 +147,24 @@ linux_proc_pid_has_state (pid_t pid, const char *state)
 int
 linux_proc_pid_is_stopped (pid_t pid)
 {
-  return linux_proc_pid_has_state (pid, "T (stopped)");
+  return linux_proc_pid_has_state (pid, "T (stopped)", 1);
+}
+
+/* Return non-zero if PID is a zombie.  If WARN, warn on failure to
+   open the /proc file.  */
+
+static int
+linux_proc_pid_is_zombie_maybe_warn (pid_t pid, int warn)
+{
+  return linux_proc_pid_has_state (pid, "Z (zombie)", warn);
+}
+
+/* See linux-procfs.h declaration.  */
+
+int
+linux_proc_pid_is_zombie_nowarn (pid_t pid)
+{
+  return linux_proc_pid_is_zombie_maybe_warn (pid, 0);
 }
 
 /* See linux-procfs.h declaration.  */
@@ -111,7 +172,7 @@ linux_proc_pid_is_stopped (pid_t pid)
 int
 linux_proc_pid_is_zombie (pid_t pid)
 {
-  return linux_proc_pid_has_state (pid, "Z (zombie)");
+  return linux_proc_pid_is_zombie_maybe_warn (pid, 1);
 }
 
 /* See linux-procfs.h declaration.  */
@@ -132,3 +193,61 @@ linux_proc_pid_get_ns (pid_t pid, const char *ns)
 
   return NULL;
 }
+
+/* See linux-procfs.h.  */
+
+void
+linux_proc_attach_tgid_threads (pid_t pid,
+				linux_proc_attach_lwp_func attach_lwp)
+{
+  DIR *dir;
+  char pathname[128];
+  int new_threads_found;
+  int iterations;
+
+  if (linux_proc_get_tgid (pid) != pid)
+    return;
+
+  xsnprintf (pathname, sizeof (pathname), "/proc/%ld/task", (long) pid);
+  dir = opendir (pathname);
+  if (dir == NULL)
+    {
+      warning (_("Could not open /proc/%ld/task.\n"), (long) pid);
+      return;
+    }
+
+  /* Scan the task list for existing threads.  While we go through the
+     threads, new threads may be spawned.  Cycle through the list of
+     threads until we have done two iterations without finding new
+     threads.  */
+  for (iterations = 0; iterations < 2; iterations++)
+    {
+      struct dirent *dp;
+
+      new_threads_found = 0;
+      while ((dp = readdir (dir)) != NULL)
+	{
+	  unsigned long lwp;
+
+	  /* Fetch one lwp.  */
+	  lwp = strtoul (dp->d_name, NULL, 10);
+	  if (lwp != 0)
+	    {
+	      ptid_t ptid = ptid_build (pid, lwp, 0);
+
+	      if (attach_lwp (ptid))
+		new_threads_found = 1;
+	    }
+	}
+
+      if (new_threads_found)
+	{
+	  /* Start over.  */
+	  iterations = -1;
+	}
+
+      rewinddir (dir);
+    }
+
+  closedir (dir);
+}
diff --git a/gdb/nat/linux-procfs.h b/gdb/nat/linux-procfs.h
index bc2fe2e..6f2a404 100644
--- a/gdb/nat/linux-procfs.h
+++ b/gdb/nat/linux-procfs.h
@@ -22,28 +22,50 @@
 #include <unistd.h>
 
 /* Return the TGID of LWPID from /proc/pid/status.  Returns -1 if not
-   found.  */
+   found.  Failure to open the /proc file results in a warning.  */
 
 extern int linux_proc_get_tgid (pid_t lwpid);
 
-/* Return the TracerPid of LWPID from /proc/pid/status.  Returns -1 if not
-   found.  */
+/* Return the TracerPid of LWPID from /proc/pid/status.  Returns -1 if
+   not found.  Does not warn on failure to open the /proc file.  */
 
-extern pid_t linux_proc_get_tracerpid (pid_t lwpid);
+extern pid_t linux_proc_get_tracerpid_nowarn (pid_t lwpid);
 
 /* Detect `T (stopped)' in `/proc/PID/status'.
    Other states including `T (tracing stop)' are reported as false.  */
 
 extern int linux_proc_pid_is_stopped (pid_t pid);
 
-/* Return non-zero if PID is a zombie.  */
+/* Return non-zero if PID is a zombie.  Failure to open the
+   /proc/pid/status file results in a warning.  */
 
 extern int linux_proc_pid_is_zombie (pid_t pid);
 
+/* Return non-zero if PID is a zombie.  Does not warn on failure to
+   open the /proc file.  */
+
+extern int linux_proc_pid_is_zombie_nowarn (pid_t pid);
+
+/* Return non-zero if /proc/PID/status indicates that PID is gone
+   (i.e., in Z/Zombie or X/Dead state).  Failure to open the /proc
+   file is assumed to indicate the thread is gone.  */
+
+extern int linux_proc_pid_is_gone (pid_t pid);
+
 /* Return an opaque string identifying PID's NS namespace or NULL if
  * the information is unavailable.  The returned string must be
  * released with xfree.  */
 
 extern char *linux_proc_pid_get_ns (pid_t pid, const char *ns);
 
+/* Callback function for linux_proc_attach_tgid_threads.  If the PTID
+   thread is not yet known, try to attach to it and return true,
+   otherwise return false.  */
+typedef int (*linux_proc_attach_lwp_func) (ptid_t ptid);
+
+/* If PID is a tgid, scan the /proc/PID/task/ directory for existing
+   threads, and call FUNC for each thread found.  */
+extern void linux_proc_attach_tgid_threads (pid_t pid,
+					    linux_proc_attach_lwp_func func);
+
 #endif /* COMMON_LINUX_PROCFS_H */
diff --git a/gdb/nat/linux-ptrace.c b/gdb/nat/linux-ptrace.c
index 9eac8ff..3981592 100644
--- a/gdb/nat/linux-ptrace.c
+++ b/gdb/nat/linux-ptrace.c
@@ -43,18 +43,44 @@ linux_ptrace_attach_fail_reason (pid_t pid, struct buffer *buffer)
 {
   pid_t tracerpid;
 
-  tracerpid = linux_proc_get_tracerpid (pid);
+  tracerpid = linux_proc_get_tracerpid_nowarn (pid);
   if (tracerpid > 0)
     buffer_xml_printf (buffer, _("process %d is already traced "
 				 "by process %d"),
 		       (int) pid, (int) tracerpid);
 
-  if (linux_proc_pid_is_zombie (pid))
+  if (linux_proc_pid_is_zombie_nowarn (pid))
     buffer_xml_printf (buffer, _("process %d is a zombie "
 				 "- the process has already terminated"),
 		       (int) pid);
 }
 
+/* See linux-ptrace.h.  */
+
+char *
+linux_ptrace_attach_fail_reason_string (ptid_t ptid, int err)
+{
+  static char *reason_string;
+  struct buffer buffer;
+  char *warnings;
+  long lwpid = ptid_get_lwp (ptid);
+
+  xfree (reason_string);
+
+  buffer_init (&buffer);
+  linux_ptrace_attach_fail_reason (lwpid, &buffer);
+  buffer_grow_str0 (&buffer, "");
+  warnings = buffer_finish (&buffer);
+  if (warnings[0] != '\0')
+    reason_string = xstrprintf ("%s (%d), %s",
+				strerror (err), err, warnings);
+  else
+    reason_string = xstrprintf ("%s (%d)",
+				strerror (err), err);
+  xfree (warnings);
+  return reason_string;
+}
+
 #if defined __i386__ || defined __x86_64__
 
 /* Address of the 'ret' instruction in asm code block below.  */
@@ -508,7 +534,8 @@ linux_disable_event_reporting (pid_t pid)
 static int
 ptrace_supports_feature (int ptrace_options)
 {
-  gdb_assert (current_ptrace_options >= 0);
+  if (current_ptrace_options == -1)
+    linux_check_ptrace_features ();
 
   return ((current_ptrace_options & ptrace_options) == ptrace_options);
 }
diff --git a/gdb/nat/linux-ptrace.h b/gdb/nat/linux-ptrace.h
index b6fd7fc..137b61a 100644
--- a/gdb/nat/linux-ptrace.h
+++ b/gdb/nat/linux-ptrace.h
@@ -89,6 +89,14 @@ struct buffer;
 #endif
 
 extern void linux_ptrace_attach_fail_reason (pid_t pid, struct buffer *buffer);
+
+/* Find all possible reasons we could have failed to attach to PTID
+   and return them as a string.  ERR is the error PTRACE_ATTACH failed
+   with (an errno).  The result is stored in a static buffer.  This
+   string should be copied into a buffer by the client if the string
+   will not be immediately used, or if it must persist.  */
+extern char *linux_ptrace_attach_fail_reason_string (ptid_t ptid, int err);
+
 extern void linux_ptrace_init_warnings (void);
 extern void linux_enable_event_reporting (pid_t pid, int attached);
 extern void linux_disable_event_reporting (pid_t pid);