Unify Solaris procfs and largefile handling

GDB currently doesn't build on 32-bit Solaris:

* On Solaris 11.4/x86:

In file included from /usr/include/sys/procfs.h:26,
                 from /vol/src/gnu/gdb/hg/master/dist/gdb/i386-sol2-nat.c:24:
/usr/include/sys/old_procfs.h:31:2: error: #error "Cannot use procfs in the large file compilation environment"
 #error "Cannot use procfs in the large file compilation environment"
  ^~~~~

* On Solaris 11.3/x86 there are several more instances of this.

The interaction between procfs and large-file support historically has
been a royal mess on Solaris:

* There are two versions of the procfs interface:

** The old ioctl-based /proc, deprecated and not used any longer in
   either gdb or binutils.

** The `new' (introduced in Solaris 2.6, 1997) structured /proc.

* There are two headers one can possibly include:

** <procfs.h> which only provides the structured /proc, definining
   _STRUCTURED_PROC=1 and then including ...

** <sys/procfs.h> which defaults to _STRUCTURED_PROC=0, the ioctl-based
   /proc, but provides structured /proc if _STRUCTURED_PROC == 1.

* procfs and the large-file environment didn't go well together:

** Until Solaris 11.3, <sys/procfs.h> would always #error in 32-bit
   compilations when the large-file environment was active
   (_FILE_OFFSET_BITS == 64).

** In both Solaris 11.4 and Illumos, this restriction was lifted for
   structured /proc.

So one has to be careful always to define _STRUCTURED_PROC=1 when
testing for or using <sys/procfs.h> on Solaris.  As the errors above
show, this isn't always the case in binutils-gdb right now.

Also one may need to disable large-file support for 32-bit compilations
on Solaris.  config/largefile.m4 meant to do this by wrapping the
AC_SYS_LARGEFILE autoconf macro with appropriate checks, yielding
ACX_LARGEFILE.  Unfortunately the macro doesn't always succeed because
it neglects the _STRUCTURED_PROC part.

To make things even worse, since GCC 9 g++ predefines
_FILE_OFFSET_BITS=64 on Solaris.  So even if largefile.m4 deciced not to
enable large-file support, this has no effect, breaking the gdb build.

This patch addresses all this as follows:

* All tests for the <sys/procfs.h> header are made with
  _STRUCTURED_PROC=1, the definition going into the various config.h
  files instead of having to make them (and sometimes failing) in the
  affected sources.

* To cope with the g++ predefine of _FILE_OFFSET_BITS=64,
  -U_FILE_OFFSET_BITS is added to various *_CPPFLAGS variables.  It had
  been far easier to have just

  #undef _FILE_OFFSET_BITS

  in config.h, but unfortunately such a construct in config.in is
  commented by config.status irrespective of indentation and whitespace
  if large-file support is disabled.  I found no way around this and
  putting the #undef in several global headers for bfd, binutils, ld,
  and gdb seemed way more invasive.

* Last, the applicability check in largefile.m4 was modified only to
  disable largefile support if really needed.  To do so, it checks if
  <sys/procfs.h> compiles with _FILE_OFFSET_BITS=64 defined.  If it
  doesn't, the disabling only happens if gdb exists in-tree and isn't
  disabled, otherwise (building binutils from a tarball), there's no
  conflict.

  What initially confused me was the check for $plugins here, which
  originally caused the disabling not to take place.  Since AC_PLUGINGS
  does enable plugin support if <dlfcn.h> exists (which it does on
  Solaris), the disabling never happened.

  I could find no explanation why the linker plugin needs large-file
  support but thought it would be enough if gld and GCC's lto-plugin
  agreed on the _FILE_OFFSET_BITS value.  Unfortunately, that's not
  enough: lto-plugin uses the simple-object interface from libiberty,
  which includes off_t arguments.  So to fully disable large-file
  support would mean also disabling it in libiberty and its users: gcc
  and libstdc++-v3.  This seems highly undesirable, so I decided to
  disable the linker plugin instead if large-file support won't work.

The patch allows binutils+gdb to build on i386-pc-solaris2.11 (both
Solaris 11.3 and 11.4, using GCC 9.3.0 which is the worst case due to
predefined _FILE_OFFSET_BITS=64).  Also regtested on
amd64-pc-solaris2.11 (again on Solaris 11.3 and 11.4),
x86_64-pc-linux-gnu and i686-pc-linux-gnu.

	config:
	* largefile.m4 (ACX_LARGEFILE) <sparc-*-solaris*|i?86-*-solaris*>:
	Check for <sys/procfs.h> incompatilibity with large-file support
	on Solaris.
	Only disable large-file support and perhaps plugins if needed.
	Set, substitute LARGEFILE_CPPFLAGS if so.

	bfd:
	* bfd.m4 (BFD_SYS_PROCFS_H): New macro.
	(BFD_HAVE_SYS_PROCFS_TYPE): Require BFD_SYS_PROCFS_H.
	Don't define _STRUCTURED_PROC.
	(BFD_HAVE_SYS_PROCFS_TYPE_MEMBER): Likewise.
	* elf.c [HAVE_SYS_PROCFS_H] (_STRUCTURED_PROC): Don't define.
	* configure.ac: Use BFD_SYS_PROCFS_H to check for <sys/procfs.h>.
	* configure, config.in: Regenerate.
	* Makefile.am (AM_CPPFLAGS): Add LARGEFILE_CPPFLAGS.
	* Makefile.in, doc/Makefile.in: Regenerate.

	binutils:
	* Makefile.am (AM_CPPFLAGS): Add LARGEFILE_CPPFLAGS.
	* Makefile.in, doc/Makefile.in: Regenerate.
	* configure: Regenerate.

	gas:
	* Makefile.am (AM_CPPFLAGS): Add LARGEFILE_CPPFLAGS.
	* Makefile.in, doc/Makefile.in: Regenerate.
	* configure: Regenerate.

	gdb:
	* proc-api.c (_STRUCTURED_PROC): Don't define.
	* proc-events.c: Likewise.
	* proc-flags.c: Likewise.
	* proc-why.c: Likewise.
	* procfs.c: Likewise.

	* Makefile.in (INTERNAL_CPPFLAGS): Add LARGEFILE_CPPFLAGS.
	* configure, config.in: Regenerate.

	gdbserver:
	* configure, config.in: Regenerate.

	gdbsupport:
	* Makefile.am (AM_CPPFLAGS): Add LARGEFILE_CPPFLAGS.
	* common.m4 (GDB_AC_COMMON): Use BFD_SYS_PROCFS_H to check for
	<sys/procfs.h>.
	* Makefile.in: Regenerate.
	* configure, config.in: Regenerate.

	gnulib:
	* configure.ac: Run ACX_LARGEFILE before gl_EARLY.
	* configure: Regenerate.

	gprof:
	* Makefile.am (AM_CPPFLAGS): Add LARGEFILE_CPPFLAGS.
	* Makefile.in: Regenerate.
	* configure: Regenerate.

	ld:
	* Makefile.am (AM_CPPFLAGS): Add LARGEFILE_CPPFLAGS.
	* Makefile.in: Regenerate.
	* configure: Regenerate.

1 files changed, 10 insertions, 6 deletions

diff --git a/bfd/bfd.m4 b/bfd/bfd.m4
index d5c3b61..21c5590 100644
--- a/bfd/bfd.m4
+++ b/bfd/bfd.m4
@@ -17,15 +17,20 @@ dnl along with this program; see the file COPYING3.  If not see
 dnl <http://www.gnu.org/licenses/>.
 dnl
 
+dnl Check for sys/procfs.h, enforcing structured /proc on Solaris.
+
+AC_DEFUN([BFD_SYS_PROCFS_H],
+[AC_DEFINE(_STRUCTURED_PROC, 1, [Use structured /proc on Solaris.])
+ AC_CHECK_HEADERS(sys/procfs.h)])
+
 dnl Check for existence of a type $1 in sys/procfs.h
 
 AC_DEFUN([BFD_HAVE_SYS_PROCFS_TYPE],
-[AC_MSG_CHECKING([for $1 in sys/procfs.h])
+[AC_REQUIRE([BFD_SYS_PROCFS_H])
+ AC_MSG_CHECKING([for $1 in sys/procfs.h])
  AC_CACHE_VAL(bfd_cv_have_sys_procfs_type_$1,
    [AC_TRY_COMPILE([
 #define _SYSCALL32
-/* Needed for new procfs interface on sparc-solaris.  */
-#define _STRUCTURED_PROC 1
 #include <sys/procfs.h>],
       [$1 avar],
       bfd_cv_have_sys_procfs_type_$1=yes,
@@ -41,12 +46,11 @@ AC_DEFUN([BFD_HAVE_SYS_PROCFS_TYPE],
 dnl Check for existence of member $2 in type $1 in sys/procfs.h
 
 AC_DEFUN([BFD_HAVE_SYS_PROCFS_TYPE_MEMBER],
-[AC_MSG_CHECKING([for $1.$2 in sys/procfs.h])
+[AC_REQUIRE([BFD_SYS_PROCFS_H])
+ AC_MSG_CHECKING([for $1.$2 in sys/procfs.h])
  AC_CACHE_VAL(bfd_cv_have_sys_procfs_type_member_$1_$2,
    [AC_TRY_COMPILE([
 #define _SYSCALL32
-/* Needed for new procfs interface on sparc-solaris.  */
-#define _STRUCTURED_PROC 1
 #include <sys/procfs.h>],
       [$1 avar; void* aref = (void*) &avar.$2],
       bfd_cv_have_sys_procfs_type_member_$1_$2=yes,