Avoid cancellable I/O primitives in ld.so.

Neither the <dlfcn.h> entry points, nor lazy symbol resolution, nor
initial shared library load-up, are cancellation points, so ld.so
should exclusively use I/O primitives that are not cancellable.  We
currently achieve this by having the cancellation hooks compile as
no-ops when IS_IN(rtld); this patch changes to using exclusively
_nocancel primitives in the source code instead, which makes the
intent clearer and significantly reduces the amount of code compiled
under IS_IN(rtld) as well as IS_IN(libc) -- in particular,
elf/Makefile no longer thinks we require a copy of unwind.c in
rtld-libc.a.  (The older mechanism is preserved as a backstop.)

The bulk of the change is splitting up the files that define the
_nocancel I/O functions, so they don't also define the variants that
*are* cancellation points; after which, the existing logic for picking
out the bits of libc that need to be recompiled as part of ld.so Just
Works.  I did this for all of the _nocancel functions, not just the
ones used by ld.so, for consistency.

fcntl was a little tricky because it's only a cancellation point for
certain opcodes (F_SETLKW(64), which can block), and the existing
__fcntl_nocancel wasn't applying the FCNTL_ADJUST_CMD hook, which
strikes me as asking for trouble, especially as the only nontrivial
definition of FCNTL_ADJUST_CMD (for powerpc64) changes F_*LK* opcodes.
To fix this, fcntl_common moves to fcntl_nocancel.c along with
__fcntl_nocancel, and changes its name to the extern (but hidden)
symbol __fcntl_nocancel_adjusted, so that regular fcntl can continue
calling it.  __fcntl_nocancel now applies FCNTL_ADJUST_CMD; so that
both both fcntl.c and fcntl_nocancel.c can see it, the only nontrivial
definition moves from sysdeps/u/s/l/powerpc/powerpc64/fcntl.c to
.../powerpc64/sysdep.h and becomes entirely a macro, instead of a macro
that calls an inline function.

The nptl version of libpthread also changes a little, because its
"compat-routines" formerly included files that defined all the
_nocancel functions it uses; instead of continuing to duplicate them,
I exported the relevant ones from libc.so as GLIBC_PRIVATE.  Since the
Linux fcntl.c calls a function defined by fcntl_nocancel.c, it can no
longer be used from libpthread.so; instead, introduce a custom
forwarder, pt-fcntl.c, and export __libc_fcntl from libc.so as
GLIBC_PRIVATE.  The nios2-linux ABI doesn't include a copy of vfork()
in libpthread, and it was handling that by manipulating
libpthread-routines in .../linux/nios2/Makefile; it is cleaner to do
what other such ports do, and have a pt-vfork.S that defines no symbols.

Right now, it appears that Hurd does not implement _nocancel I/O, so
sysdeps/generic/not-cancel.h will forward everything back to the
regular functions.  This changed the names of some of the functions
that sysdeps/mach/hurd/dl-sysdep.c needs to interpose.

	* elf/dl-load.c, elf/dl-misc.c, elf/dl-profile.c, elf/rtld.c
	* sysdeps/unix/sysv/linux/dl-sysdep.c
	Include not-cancel.h.  Use __close_nocancel instead of __close,
	__open64_nocancel instead of __open, __read_nocancel instead of
	__libc_read, and __write_nocancel instead of __libc_write.

	* csu/check_fds.c (check_one_fd)
	* sysdeps/posix/fdopendir.c (__fdopendir)
	* sysdeps/posix/opendir.c (__alloc_dir): Use __fcntl_nocancel
        instead of __fcntl and/or __libc_fcntl.

	* sysdeps/unix/sysv/linux/pthread_setname.c (pthread_setname_np)
	* sysdeps/unix/sysv/linux/pthread_getname.c (pthread_getname_np)
        * sysdeps/unix/sysv/linux/i386/smp.h (is_smp_system):
	Use __open64_nocancel instead of __open_nocancel.

	* sysdeps/unix/sysv/linux/not-cancel.h: Move all of the
	hidden_proto declarations to the end and issue them if either
	IS_IN(libc) or IS_IN(rtld).
	* sysdeps/unix/sysv/linux/Makefile [subdir=io] (sysdep_routines):
	Add close_nocancel, fcntl_nocancel, nanosleep_nocancel,
	open_nocancel, open64_nocancel, openat_nocancel, pause_nocancel,
	read_nocancel, waitpid_nocancel, write_nocancel.

        * io/Versions [GLIBC_PRIVATE]: Add __libc_fcntl,
        __fcntl_nocancel, __open64_nocancel, __write_nocancel.
        * posix/Versions: Add __nanosleep_nocancel, __pause_nocancel.

        * nptl/pt-fcntl.c: New file.
        * nptl/Makefile (pthread-compat-wrappers): Remove fcntl.
        (libpthread-routines): Add pt-fcntl.
        * include/fcntl.h (__fcntl_nocancel_adjusted): New function.
        (__libc_fcntl): Remove attribute_hidden.
	* sysdeps/unix/sysv/linux/fcntl.c (__libc_fcntl): Call
	__fcntl_nocancel_adjusted, not fcntl_common.
        (__fcntl_nocancel): Move to new file fcntl_nocancel.c.
	(fcntl_common): Rename to __fcntl_nocancel_adjusted; also move
	to fcntl_nocancel.c.
	* sysdeps/unix/sysv/linux/fcntl_nocancel.c: New file.
	* sysdeps/unix/sysv/linux/powerpc/powerpc64/fcntl.c: Remove file.
	* sysdeps/unix/sysv/linux/powerpc/powerpc64/sysdep.h:
	Define FCNTL_ADJUST_CMD here, as a self-contained macro.

	* sysdeps/unix/sysv/linux/close.c: Move __close_nocancel to...
	* sysdeps/unix/sysv/linux/close_nocancel.c: ...this new file.
	* sysdeps/unix/sysv/linux/nanosleep.c: Move __nanosleep_nocancel to...
	* sysdeps/unix/sysv/linux/nanosleep_nocancel.c: ...this new file.
	* sysdeps/unix/sysv/linux/open.c: Move __open_nocancel to...
	* sysdeps/unix/sysv/linux/open_nocancel.c: ...this new file.
	* sysdeps/unix/sysv/linux/open64.c: Move __open64_nocancel to...
	* sysdeps/unix/sysv/linux/open64_nocancel.c: ...this new file.
	* sysdeps/unix/sysv/linux/openat.c: Move __openat_nocancel to...
	* sysdeps/unix/sysv/linux/openat_nocancel.c: ...this new file.
	* sysdeps/unix/sysv/linux/openat64.c: Move __openat64_nocancel to...
	* sysdeps/unix/sysv/linux/openat64_nocancel.c: ...this new file.
	* sysdeps/unix/sysv/linux/pause.c: Move __pause_nocancel to...
	* sysdeps/unix/sysv/linux/pause_nocancel.c: ...this new file.
	* sysdeps/unix/sysv/linux/read.c: Move __read_nocancel to...
	* sysdeps/unix/sysv/linux/read_nocancel.c: ...this new file.
	* sysdeps/unix/sysv/linux/waitpid.c: Move __waitpid_nocancel to...
	* sysdeps/unix/sysv/linux/waitpid_nocancel.c: ...this new file.
	* sysdeps/unix/sysv/linux/write.c: Move __write_nocancel to...
	* sysdeps/unix/sysv/linux/write_nocancel.c: ...this new file.

        * sysdeps/unix/sysv/linux/nios2/Makefile: Don't override
        libpthread-routines.
        * sysdeps/unix/sysv/linux/nios2/pt-vfork.S: New file which
        defines nothing.

        * sysdeps/mach/hurd/dl-sysdep.c: Define __read instead of
        __libc_read, and __write instead of __libc_write.  Define
        __open64 in addition to __open.

4 files changed, 30 insertions, 28 deletions

diff --git a/elf/dl-load.c b/elf/dl-load.c
index 4312369..c52d011 100644
--- a/elf/dl-load.c
+++ b/elf/dl-load.c
@@ -44,7 +44,7 @@
 #include <dl-unmap-segments.h>
 #include <dl-machine-reject-phdr.h>
 #include <dl-sysdep-open.h>
-
+#include <not-cancel.h>
 
 #include <endian.h>
 #if BYTE_ORDER == BIG_ENDIAN
@@ -776,7 +776,7 @@ lose (int code, int fd, const char *name, char *realname, struct link_map *l,
 {
   /* The file might already be closed.  */
   if (fd != -1)
-    (void) __close (fd);
+    (void) __close_nocancel (fd);
   if (l != NULL && l->l_origin != (char *) -1l)
     free ((char *) l->l_origin);
   free (l);
@@ -835,7 +835,7 @@ _dl_map_object_from_fd (const char *name, const char *origname, int fd,
       {
 	/* The object is already loaded.
 	   Just bump its reference count and return it.  */
-	__close (fd);
+	__close_nocancel (fd);
 
 	/* If the name is not in the list of names for this object add
 	   it.  */
@@ -863,7 +863,7 @@ _dl_map_object_from_fd (const char *name, const char *origname, int fd,
 
       /* No need to bump the refcount of the real object, ld.so will
 	 never be unloaded.  */
-      __close (fd);
+      __close_nocancel (fd);
 
       /* Add the map for the mirrored object to the object list.  */
       _dl_add_to_namespace_list (l, nsid);
@@ -877,7 +877,7 @@ _dl_map_object_from_fd (const char *name, const char *origname, int fd,
       /* We are not supposed to load the object unless it is already
 	 loaded.  So return now.  */
       free (realname);
-      __close (fd);
+      __close_nocancel (fd);
       return NULL;
     }
 
@@ -896,7 +896,7 @@ _dl_map_object_from_fd (const char *name, const char *origname, int fd,
       if (_dl_zerofd == -1)
 	{
 	  free (realname);
-	  __close (fd);
+	  __close_nocancel (fd);
 	  _dl_signal_error (errno, NULL, NULL,
 			    N_("cannot open zero fill device"));
 	}
@@ -962,7 +962,7 @@ _dl_map_object_from_fd (const char *name, const char *origname, int fd,
     {
       phdr = alloca (maplength);
       __lseek (fd, header->e_phoff, SEEK_SET);
-      if ((size_t) __libc_read (fd, (void *) phdr, maplength) != maplength)
+      if ((size_t) __read_nocancel (fd, (void *) phdr, maplength) != maplength)
 	{
 	  errstring = N_("cannot read file data");
 	  goto call_lose_errno;
@@ -1232,7 +1232,7 @@ cannot enable executable stack as shared object requires");
     l->l_tls_initimage = (char *) l->l_tls_initimage + l->l_addr;
 
   /* We are done mapping in the file.  We no longer need the descriptor.  */
-  if (__glibc_unlikely (__close (fd) != 0))
+  if (__glibc_unlikely (__close_nocancel (fd) != 0))
     {
       errstring = N_("cannot close file descriptor");
       goto call_lose_errno;
@@ -1447,7 +1447,7 @@ open_verify (const char *name, int fd,
         {
           /* An audit library changed what we're supposed to open,
              so FD no longer matches it.  */
-          __close (fd);
+          __close_nocancel (fd);
           fd = -1;
         }
     }
@@ -1455,7 +1455,7 @@ open_verify (const char *name, int fd,
 
   if (fd == -1)
     /* Open the file.  We always open files read-only.  */
-    fd = __open (name, O_RDONLY | O_CLOEXEC);
+    fd = __open64_nocancel (name, O_RDONLY | O_CLOEXEC);
 
   if (fd != -1)
     {
@@ -1474,8 +1474,8 @@ open_verify (const char *name, int fd,
       /* Read in the header.  */
       do
 	{
-	  ssize_t retlen = __libc_read (fd, fbp->buf + fbp->len,
-					sizeof (fbp->buf) - fbp->len);
+	  ssize_t retlen = __read_nocancel (fd, fbp->buf + fbp->len,
+					    sizeof (fbp->buf) - fbp->len);
 	  if (retlen <= 0)
 	    break;
 	  fbp->len += retlen;
@@ -1598,7 +1598,8 @@ open_verify (const char *name, int fd,
 	{
 	  phdr = alloca (maplength);
 	  __lseek (fd, ehdr->e_phoff, SEEK_SET);
-	  if ((size_t) __libc_read (fd, (void *) phdr, maplength) != maplength)
+	  if ((size_t) __read_nocancel (fd, (void *) phdr, maplength)
+	      != maplength)
 	    {
 	    read_error:
 	      errval = errno;
@@ -1648,7 +1649,7 @@ open_verify (const char *name, int fd,
 		    abi_note = abi_note_malloced;
 		  }
 		__lseek (fd, ph->p_offset, SEEK_SET);
-		if (__libc_read (fd, (void *) abi_note, size) != size)
+		if (__read_nocancel (fd, (void *) abi_note, size) != size)
 		  {
 		    free (abi_note_malloced);
 		    goto read_error;
@@ -1680,7 +1681,7 @@ open_verify (const char *name, int fd,
 		|| (GLRO(dl_osversion) && GLRO(dl_osversion) < osversion))
 	      {
 	      close_and_out:
-		__close (fd);
+		__close_nocancel (fd);
 		__set_errno (ENOENT);
 		fd = -1;
 	      }
@@ -1797,7 +1798,7 @@ open_path (const char *name, size_t namelen, int mode,
 		  /* The shared object cannot be tested for being SUID
 		     or this bit is not set.  In this case we must not
 		     use this object.  */
-		  __close (fd);
+		  __close_nocancel (fd);
 		  fd = -1;
 		  /* We simply ignore the file, signal this by setting
 		     the error value which would have been set by `open'.  */
@@ -1818,7 +1819,7 @@ open_path (const char *name, size_t namelen, int mode,
 	    {
 	      /* No memory for the name, we certainly won't be able
 		 to load and link it.  */
-	      __close (fd);
+	      __close_nocancel (fd);
 	      return -1;
 	    }
 	}
diff --git a/elf/dl-misc.c b/elf/dl-misc.c
index b717499..2eb81ee 100644
--- a/elf/dl-misc.c
+++ b/elf/dl-misc.c
@@ -33,7 +33,7 @@
 #include <sysdep.h>
 #include <_itoa.h>
 #include <dl-writev.h>
-
+#include <not-cancel.h>
 
 /* Read the whole contents of FILE into new mmap'd space with given
    protections.  *SIZEP gets the size of the file.  On error MAP_FAILED
@@ -44,7 +44,7 @@ _dl_sysdep_read_whole_file (const char *file, size_t *sizep, int prot)
 {
   void *result = MAP_FAILED;
   struct stat64 st;
-  int fd = __open (file, O_RDONLY | O_CLOEXEC);
+  int fd = __open64_nocancel (file, O_RDONLY | O_CLOEXEC);
   if (fd >= 0)
     {
       if (__fxstat64 (_STAT_VER, fd, &st) >= 0)
@@ -65,7 +65,7 @@ _dl_sysdep_read_whole_file (const char *file, size_t *sizep, int prot)
 #endif
 			     , fd, 0);
 	}
-      __close (fd);
+      __close_nocancel (fd);
     }
   return result;
 }
diff --git a/elf/dl-profile.c b/elf/dl-profile.c
index 7f15352..bac3718 100644
--- a/elf/dl-profile.c
+++ b/elf/dl-profile.c
@@ -35,6 +35,7 @@
 #include <sys/param.h>
 #include <sys/stat.h>
 #include <atomic.h>
+#include <not-cancel.h>
 
 /* The LD_PROFILE feature has to be implemented different to the
    normal profiling using the gmon/ functions.  The problem is that an
@@ -324,7 +325,7 @@ _dl_start_profile (void)
   *cp++ = '/';
   __stpcpy (__stpcpy (cp, GLRO(dl_profile)), ".profile");
 
-  fd = __open (filename, O_RDWR | O_CREAT | O_NOFOLLOW, DEFFILEMODE);
+  fd = __open64_nocancel (filename, O_RDWR|O_CREAT|O_NOFOLLOW, DEFFILEMODE);
   if (fd == -1)
     {
       char buf[400];
@@ -335,7 +336,7 @@ _dl_start_profile (void)
     print_error:
       errnum = errno;
       if (fd != -1)
-	__close (fd);
+	__close_nocancel (fd);
       _dl_error_printf (errstr, filename,
 			__strerror_r (errnum, buf, sizeof buf));
       return;
@@ -364,15 +365,14 @@ _dl_start_profile (void)
 	  goto print_error;
 	}
 
-      if (TEMP_FAILURE_RETRY (__libc_write (fd, buf, (expected_size
-						      & (GLRO(dl_pagesize)
-							 - 1))))
+      if (TEMP_FAILURE_RETRY
+	  (__write_nocancel (fd, buf, (expected_size & (GLRO(dl_pagesize) - 1))))
 	  < 0)
 	goto cannot_create;
     }
   else if (st.st_size != expected_size)
     {
-      __close (fd);
+      __close_nocancel (fd);
     wrong_format:
 
       if (addr != NULL)
@@ -392,7 +392,7 @@ _dl_start_profile (void)
     }
 
   /* We don't need the file descriptor anymore.  */
-  __close (fd);
+  __close_nocancel (fd);
 
   /* Pointer to data after the header.  */
   hist = (char *) (addr + 1);
diff --git a/elf/rtld.c b/elf/rtld.c
index e7681eb..8c732ad 100644
--- a/elf/rtld.c
+++ b/elf/rtld.c
@@ -41,6 +41,7 @@
 #include <tls.h>
 #include <stap-probe.h>
 #include <stackinfo.h>
+#include <not-cancel.h>
 
 #include <assert.h>
 
@@ -2673,7 +2674,7 @@ process_envvars (enum mode *modep)
       *--startp = '.';
       startp = memcpy (startp - name_len, debug_output, name_len);
 
-      GLRO(dl_debug_fd) = __open (startp, flags, DEFFILEMODE);
+      GLRO(dl_debug_fd) = __open64_nocancel (startp, flags, DEFFILEMODE);
       if (GLRO(dl_debug_fd) == -1)
 	/* We use standard output if opening the file failed.  */
 	GLRO(dl_debug_fd) = STDOUT_FILENO;