This returns an int, not a long int or an ssize_t (as one test was
inconsistently assuming).

libctf/ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* testsuite/libctf-lookup/struct-iteration.c (main):
	ctf_member_count returns an int.

We never actually check to see if the compiler supports CTF,
or even if a suitable compiler exists.

libctf/ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* Makefile.am (BASEDIR): New.
	(BFDDIR): Likewise.
	(check-DEJAGNU): Add development.exp to prerequisites.
	(development.exp): New.
	(CONFIG_STATUS_DEPENDENCIES): New.
	(EXTRA_DEJAGNU_SITE_CONFIG): Likewise.
	(DISTCLEANFILES): Likewise.
	* Makefile.in: Regenerated.
	* testsuite/lib/ctf-lib.exp (check_ctf_available): Return boolean.
	* testsuite/libctf-lookup/lookup.exp: Call check_ctf_available.
	* testsuite/libctf-regression/regression.exp: Likewise.

The type printer was unconditionally printing these as if they were
forwards to structs, even if they were forwards to unions or enums.

ld/ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* testsuite/ld-ctf/enum-forward.c: New test.
	* testsuite/ld-ctf/enum-forward.c: New results.

libctf/ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-types.c (ctf_type_aname): Print forwards to unions and enums
	properly.

When you look up a type by name using ctf_lookup_by_name, in most cases
libctf can just strip off any qualifiers and look for the name, but for
pointer types this doesn't work, since the caller will want the pointer
type itself.  But pointer types are nameless, and while they cite the
types they point to, looking up a type by name requires a link going the
*other way*, from the type pointed to to the pointer type that points to
it.

libctf has always built this up at open time: ctf_ptrtab is an array of
type indexes pointing from the index of every type to the index of the
type that points to it.  But because it is built up at open time (and
because it uses type indexes and not type IDs) it is restricted to
working within a single dict and ignoring parent/child
relationships. This is normally invisible, unless you manage to get a
dict with a type in the parent but the only pointer to it in a child.
The ctf_ptrtab will not track this relationship, so lookups of this
pointer type by name will fail.  Since which type is in the parent and
which in the child is largely opaque to the user (which goes where is up
to the deduplicator, and it can and does reshuffle things to save
space), this leads to a very bad user experience, with an
obviously-visible pointer type which ctf_lookup_by_name claims doesn't
exist.

The fix is to have another array, ctf_pptrtab, which is populated in
child dicts: like the parent's ctf_ptrtab, it has one element per type
in the parent, but is all zeroes except for those types which are
pointed to by types in the child: so it maps parent dict indices to
child dict indices.  The array is grown, and new child types scanned,
whenever a lookup happens and new types have been added to the child
since the last time a lookup happened that might need the pptrtab.
(So for non-writable dicts, this only happens once, since new types
cannot be added to non-writable dicts at all.)

Since this introduces new complexity (involving updating only part of
the ctf_pptrtab) which is only seen when a writable dict is in use, we
introduce a new libctf-writable testsuite that contains lookup tests
with no corresponding CTF-containing .c files (which can thus be run
even on platforms with no .ctf-section support in the linker yet), and
add a test to check that creation of pointers in children to types in
parents and a following lookup by name works as expected.  The non-
writable case is tested in a new libctf-regression testsuite which is
used to track now-fixed outright bugs in libctf.

libctf/ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-impl.h (ctf_dict_t) <ctf_pptrtab>: New.
	<ctf_pptrtab_len>: New.
	<ctf_pptrtab_typemax>: New.
	* ctf-create.c (ctf_serialize): Update accordingly.
	(ctf_add_reftype): Note that we don't need to update pptrtab here,
	despite updating ptrtab.
	* ctf-open.c (ctf_dict_close): Destroy the pptrtab.
	(ctf_import): Likewise.
	(ctf_import_unref): Likewise.
	* ctf-lookup.c (grow_pptrtab): New.
	(refresh_pptrtab): New, update a pptrtab.
	(ctf_lookup_by_name): Turn into a wrapper around (and rename to)...
	(ctf_lookup_by_name_internal): ... this: construct the pptrtab, and
	use it in addition to the parent's ptrtab when parent dicts are
	searched.
	* testsuite/libctf-regression/regression.exp: New testsuite for
	regression tests.
	* testsuite/libctf-regression/pptrtab*: New test.
	* testsuite/libctf-writable/writable.exp: New testsuite for tests of
	writable CTF dicts.
	* testsuite/libctf-writable/pptrtab*: New test.

Parent dicts are nowadays imported automatically in most situations, so
the comment in ctf_archive_iter warning people that they need to import
parents by hand is wrong.  Remove it.

libctf/ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-archive.c (ctf_archive_iter): Remove outdated comment.

libctf has no intrinsic support for the GCC unnamed structure member
extension.  This principally means that you can't look up named members
inside unnamed struct or union members via ctf_member_info: you have to
tiresomely find out the type ID of the unnamed members via iteration,
then look in each of these.

This is ridiculous.  Fix it by extending ctf_member_info so that it
recurses into unnamed members for you: this is still unambiguous because
GCC won't let you create ambiguously-named members even in the presence
of this extension.

For consistency, and because the release hasn't happened and we can
still do this, break the ctf_member_next API and add flags: we specify
one flag, CTF_MN_RECURSE, which if set causes ctf_member_next to
automatically recurse into unnamed members for you, returning not only
the members themselves but all their contained members, so that you can
use ctf_member_next to identify every member that it would be valid to
call ctf_member_info with.

New lookup tests are added for all of this.

include/ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-api.h (CTF_MN_RECURSE): New.
	(ctf_member_next): Add flags argument.

libctf/ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-impl.h (struct ctf_next) <u.ctn_next>: Move to...
	<ctn_next>: ... here.
	* ctf-util.c (ctf_next_destroy): Unconditionally destroy it.
	* ctf-lookup.c (ctf_symbol_next): Adjust accordingly.
	* ctf-types.c (ctf_member_iter): Reimplement in terms of...
	(ctf_member_next): ... this.  Support recursive unnamed member
	iteration (off by default).
	(ctf_member_info): Look up members in unnamed sub-structs.
	* ctf-dedup.c (ctf_dedup_rhash_type): Adjust ctf_member_next call.
	(ctf_dedup_emit_struct_members): Likewise.
	* testsuite/libctf-lookup/struct-iteration-ctf.c: Test empty unnamed
	members, and a normal member after the end.
	* testsuite/libctf-lookup/struct-iteration.c: Verify that
	ctf_member_count is consistent with the number of successful returns
	from a non-recursive ctf_member_next.
	* testsuite/libctf-lookup/struct-iteration-*: New, test iteration
	over struct members.
	* testsuite/libctf-lookup/struct-lookup.c: New test.
	* testsuite/libctf-lookup/struct-lookup.lk: New test.

In the last cycle there have been various changes that have replaced
parts of the CTF format with other parts without format
compatibility.  This was not a compat break, because the old format was
never accepted by any version of libctf (the not-in-official-release CTF
compiler patch was emitting an invalid func info section), but
nonetheless it can confuse users using that patch if they link together
object files and find the func info sections in the inputs silently
disappearing.

Scan the linker inputs for this problem and emit a warning if any are
found.

libctf/ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-link.c (ctf_link_warn_outdated_inputs): New.
	(ctf_link_write): Call it.

I had reports that this doesn't work.  This test shows it working (and
also shows how annoying it is to do symbol lookup by name with the
present API: we need a ctf_arc_lookup_symbol_name for users that don't
already have a symtab handy).

libctf/ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* testsuite/libctf-lookup/enum-symbol.lk: New symbol-lookup test.
	* testsuite/libctf-lookup/enum-symbol-ctf.c: New CTF input.
	* testsuite/libctf-lookup/enum-symbol.c: New lookup test.

This introduces a new lookup testsuite under libctf, which operates by
compiling (with libtool) a "lookup" .c file that uses libctf to analyze
some other program, then compiling some number of test object files with
CTF and optionally linking them together and running the lookup program
on the test object files (or linked test binary), before diffing the
result much as run_dump_test does.

This lets us test the portions of libctf that are not previously
testable, notably the portions that do lookup on linked output and
that create dynamic dictionaries and then do lookup on them before
writing them out, something that is not tested by the ld-ctf testsuite
because the linker never does this.

A couple of simple tests are added: one testing the functionality of
enum lookups, and one testing that the recently-added commit adding
extra paranoia to incomplete type handling doesn't break linking and
that the result of the link is an (otherwise-impossible) array of
forward type in the shared CTF dict.

ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* Makefile.def (libctf): No longer no_check.  Checking depends on
	all-ld.
	* Makefile.in: Regenerated.

libctf/ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* Makefile.am (EXPECT): New.
	(RUNTEST): Likewise.
	(RUNTESTFLAGS): Likewise.
	(CC_FOR_TARGET): Likewise.
	(check-DEJAGNU): Likewise.
	(AUTOMAKE_OPTIONS): Add dejagnu.
	* Makefile.in: Regenerated.
	* testsuite/config/default.exp: New.
	* testsuite/lib/ctf-lib.exp: Likewise.
	* testsuite/libctf-lookup/enum.lk: New test.
	* testsuite/libctf-lookup/enum-ctf.c: New CTF input.
	* testsuite/libctf-lookup/enum.c: New lookup test.
	* testsuite/libctf-lookup/ambiguous-struct*.c: New test.
	* testsuite/libctf-lookup/lookup.exp: New.

This complex morass inherited from libopcodes, which endeavours to
implement the effect of specifying ../bfd/libbfd.la in _LIBADD without
actually doing so, appears to be working around a libtool bug which as
far as I can see is no longer present (i.e., the install directory no
longer appears in -L arguments in libtool link-mode invocations, so
there is no danger of picking up old libbfds or other dependent
libraries).

Replaced with a simple reference to libbfd.la in the appropriate place.
Also adjusted things a little more so that libctf.la and libctf-nobfd.la
are self-contained, even when linking statically.  This opens up the
possibility of running libtool to link against libctf from inside the
(upcoming) testsuite.

libctf/ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* configure.ac (BFD_LIBADD): Remove.
	(BFD_DEPENDENCIES): Likewise. Remove associated cases.
	(SHARED_LIBADD): Rename to...
	(CTF_LIBADD): ... this.  Stick in a suitable libiberty even when
	linking statically.
	* Makefile.am (libctf_nobfd_la_LIBADD): Adjust accordingly.
	libctf uses libintl.
	(libctf_la_LIBADD): Reference libbfd.la directly, not via
	BFD_LIBADD.
	(libctf_la_DEPENDENCIES): Remove.
	* Makefile.in: Regenerate.
	* configure: Likewise.

This commit adds dumping of enumerands in this general form:

    0x3: (kind 8) enum eleven_els (size 0x4) (aligned at 0x4)
         ELEVEN_ONE: 10
         ELEVEN_TWO: 11
         ELEVEN_THREE: -256
         ELEVEN_FOUR: -255
         ELEVEN_FIVE: -254
         ...
         ELEVEN_SEVEN: -252
         ELEVEN_EIGHT: -251
         ELEVEN_NINE: -250
         ELEVEN_TEN: -249
         ELEVEN_ELEVEN: -248

The first and last enumerands in the enumerated type are printed so that
you can tell if they've been cut off at one end or the other.  (For now,
there is no way to control how many enumerands are printed.)

The dump output in general is improved, from this sort of thing a few
days ago:

     4c: char [0x0:0x8] (size 0x1)
        [0x0] (ID 0x4c) (kind 1) char:8 (aligned at 0x1, format 0x3, offset:bits 0x0:0x8)
     4d: char * (size 0x8) -> 4c: char [0x0:0x8] (size 0x1)
        [0x0] (ID 0x4d) (kind 3) char * (aligned at 0x8)
[...]
     5a: struct _IO_FILE (size 0xd8)
        [0x0] (ID 0x5a) (kind 6) struct _IO_FILE (aligned at 0x4)
            [0x0] (ID 0x3) (kind 1) int _flags:32 (aligned at 0x4, format 0x1, offset:bits 0x0:0x20)
            [0x40] (ID 0x4d) (kind 3) char * _IO_read_ptr (aligned at 0x8)
            [0x80] (ID 0x4d) (kind 3) char * _IO_read_end (aligned at 0x8)
            [0xc0] (ID 0x4d) (kind 3) char * _IO_read_base (aligned at 0x8)
     5b: __FILE (size 0xd8) -> 5a: struct _IO_FILE (size 0xd8)
        [0x0] (ID 0x5b) (kind 10) __FILE (aligned at 0x4)
            [0x0] (ID 0x3) (kind 1) int _flags:32 (aligned at 0x4, format 0x1, offset:bits 0x0:0x20)
            [0x40] (ID 0x4d) (kind 3) char * _IO_read_ptr (aligned at 0x8)
            [0x80] (ID 0x4d) (kind 3) char * _IO_read_end (aligned at 0x8)
            [0xc0] (ID 0x4d) (kind 3) char * _IO_read_base (aligned at 0x8)
[...]
     406: struct coff_link_hash_entry (size 0x60)
        [0x0] (ID 0x406) (kind 6) struct coff_link_hash_entry (aligned at 0x8)
            [0x0] (ID 0x2b3) (kind 6) struct bfd_link_hash_entry root (aligned at 0x8)
                [0x0] (ID 0x1d6) (kind 6) struct bfd_hash_entry root (aligned at 0x8)
                    [0x0] (ID 0x1d7) (kind 3) struct bfd_hash_entry * next (aligned at 0x8)
                    [0x40] (ID 0x61) (kind 3) const char * string (aligned at 0x8)
                    [0x80] (ID 0x1) (kind 1) long unsigned int hash:64 (aligned at 0x8, format 0x0, offset:bits 0x0:0x40)
                [0xc0] (ID 0x397) (kind 8) enum bfd_link_hash_type  type:8 (aligned at 0x1, format 0x0, offset:bits 0x0:0x8)
                [0xc8] (ID 0x1c7) (kind 1) unsigned int  non_ir_ref_regular:1 (aligned at 0x1, format 0x0, offset:bits 0x8:0x1)
                [0xc9] (ID 0x1c8) (kind 1) unsigned int  non_ir_ref_dynamic:1 (aligned at 0x1, format 0x0, offset:bits 0x9:0x1)
                [0xca] (ID 0x1c9) (kind 1) unsigned int  linker_def:1 (aligned at 0x1, format 0x0, offset:bits 0xa:0x1)
                [0xcb] (ID 0x1ca) (kind 1) unsigned int  ldscript_def:1 (aligned at 0x1, format 0x0, offset:bits 0xb:0x1)
                [0xcc] (ID 0x1cb) (kind 1) unsigned int  rel_from_abs:1 (aligned at 0x1, format 0x0, offset:bits 0xc:0x1)

... to this:

    0x4c: (kind 1) char (format 0x3) (size 0x1) (aligned at 0x1)
    0x4d: (kind 3) char * (size 0x8) (aligned at 0x8) -> 0x4c: (kind 1) char (format 0x3) (size 0x1) (aligned at 0x1)
    0x5a: (kind 6) struct _IO_FILE (size 0xd8) (aligned at 0x4)
          [0x0] _flags: ID 0x3: (kind 1) int (format 0x1) (size 0x4) (aligned at 0x4)
          [0x40] _IO_read_ptr: ID 0x4d: (kind 3) char * (size 0x8) (aligned at 0x8)
          [0x80] _IO_read_end: ID 0x4d: (kind 3) char * (size 0x8) (aligned at 0x8)
          [0xc0] _IO_read_base: ID 0x4d: (kind 3) char * (size 0x8) (aligned at 0x8)
          [0x100] _IO_write_base: ID 0x4d: (kind 3) char * (size 0x8) (aligned at 0x8)
    0x5b: (kind 10) __FILE (size 0xd8) (aligned at 0x4) -> 0x5a: (kind 6) struct _IO_FILE (size 0xd8) (aligned at 0x4)
[...]
    0x406: (kind 6) struct coff_link_hash_entry (size 0x60) (aligned at 0x8)
           [0x0] root: ID 0x2b3: (kind 6) struct bfd_link_hash_entry (size 0x38) (aligned at 0x8)
               [0x0] root: ID 0x1d6: (kind 6) struct bfd_hash_entry (size 0x18) (aligned at 0x8)
                   [0x0] next: ID 0x1d7: (kind 3) struct bfd_hash_entry * (size 0x8) (aligned at 0x8)
                   [0x40] string: ID 0x61: (kind 3) const char * (size 0x8) (aligned at 0x8)
                   [0x80] hash: ID 0x1: (kind 1) long unsigned int (format 0x0) (size 0x8) (aligned at 0x8)
               [0xc0] type: ID 0x397: (kind 8) enum bfd_link_hash_type (format 0x7f2e) (size 0x1) (aligned at 0x1)
               [0xc8] non_ir_ref_regular: ID 0x1c7: (kind 1) unsigned int:1 [slice 0x8:0x1] (format 0x0) (size 0x1) (aligned at 0x1)
               [0xc9] non_ir_ref_dynamic: ID 0x1c8: (kind 1) unsigned int:1 [slice 0x9:0x1] (format 0x0) (size 0x1) (aligned at 0x1)
               [0xca] linker_def: ID 0x1c9: (kind 1) unsigned int:1 [slice 0xa:0x1] (format 0x0) (size 0x1) (aligned at 0x1)
               [0xcb] ldscript_def: ID 0x1ca: (kind 1) unsigned int:1 [slice 0xb:0x1] (format 0x0) (size 0x1) (aligned at 0x1)
               [0xcc] rel_from_abs: ID 0x1cb: (kind 1) unsigned int:1 [slice 0xc:0x1] (format 0x0) (size 0x1) (aligned at 0x1)
[...]

In particular, indented subsections are only present for actual structs
and unions, not forwards to them, and the structure itself doesn't add a
spurious level of indentation; structure field names are easier to spot
(at the cost of not making them look so much like C field declarations
any more, but they weren't always shown in valid decl syntax even before
this change) the size, type kind, and alignment are shown for all types
for which they are meaningful; bitfield info is only shown for actual
bitfields within structures and not ordinary integral fields; and type
IDs are never omitted.  Type printing is in general much more consistent
and there is much less duplicated code in the type dumper.

There is one user-visible effect outside the dumper: ctf_type_(a)name
was erroneously emitting a trailing space on the name of slice types,
even though a slice of an int and an int with the corresponding encoding
represent the same type and should have the same print form.  This
trailing space is now gone.

ld/ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* testsuite/ld-ctf/array.d: Adjust for dumper changes.
	* testsuite/ld-ctf/conflicting-cycle-1.B-1.d: Likewise.
	* testsuite/ld-ctf/conflicting-cycle-1.B-2.d: Likewise.
	* testsuite/ld-ctf/conflicting-cycle-1.parent.d: Likewise.
	* testsuite/ld-ctf/conflicting-cycle-2.A-1.d: Likewise.
	* testsuite/ld-ctf/conflicting-cycle-2.A-2.d: Likewise.
	* testsuite/ld-ctf/conflicting-cycle-2.parent.d: Likewise.
	* testsuite/ld-ctf/conflicting-cycle-3.C-1.d: Likewise.
	* testsuite/ld-ctf/conflicting-cycle-3.C-2.d: Likewise.
	* testsuite/ld-ctf/conflicting-cycle-3.parent.d: Likewise.
	* testsuite/ld-ctf/conflicting-enums.d: Likewise.
	* testsuite/ld-ctf/conflicting-typedefs.d: Likewise.
	* testsuite/ld-ctf/cross-tu-cyclic-conflicting.d: Likewise.
	* testsuite/ld-ctf/cross-tu-cyclic-nonconflicting.d: Likewise.
	* testsuite/ld-ctf/cross-tu-into-cycle.d: Likewise.
	* testsuite/ld-ctf/cross-tu-noncyclic.d: Likewise.
	* testsuite/ld-ctf/cycle-1.d: Likewise.
	* testsuite/ld-ctf/cycle-2.A.d: Likewise.
	* testsuite/ld-ctf/cycle-2.B.d: Likewise.
	* testsuite/ld-ctf/cycle-2.C.d: Likewise.
	* testsuite/ld-ctf/data-func-conflicted.d: Likewise.
	* testsuite/ld-ctf/diag-cttname-null.d: Likewise.
	* testsuite/ld-ctf/diag-cuname.d: Likewise.
	* testsuite/ld-ctf/diag-parlabel.d: Likewise.
	* testsuite/ld-ctf/diag-wrong-magic-number-mixed.d: Likewise.
	* testsuite/ld-ctf/forward.d: Likewise.
	* testsuite/ld-ctf/function.d: Likewise.
	* testsuite/ld-ctf/slice.d: Likewise.
	* testsuite/ld-ctf/super-sub-cycles.d: Likewise.
	* testsuite/ld-ctf/enums.c: New test.
	* testsuite/ld-ctf/enums.d: New test.

libctf/ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-decl.c (ctf_decl_push): Exclude slices from the decl stack.
	* ctf-types.c (ctf_type_aname): No longer deal with slices here.
	* ctf-dump.c (ctf_dump_membstate_t) <cdm_toplevel_indent>: Constify.
	(CTF_FT_REFS): New.
	(CTF_FT_BITFIELD): Likewise.
	(CTF_FT_ID): Likewise.
	(ctf_dump_member): Do not do indentation here. Migrate the
	type-printing parts of this into...
	(ctf_dump_format_type): ... here, to be shared by all type printers.
	Get the errno value for non-representable types right.  Do not print
	bitfield info for non-bitfields.  Improve the format and indentation
	of other type output.  Shuffle spacing around to make all indentation
	either 'width of column' or 4 chars.
	(ctf_dump_label): Pass CTF_FT_REFS to ctf_dump_format_type.
	(ctf_dump_objts): Likewise.  Spacing shuffle.
	(ctf_dump_var): Likewise.
	(type_hex_digits): Migrate down in the file, to above its new user.
	(ctf_dump_type): Indent here instead.  Pass CTF_FT_REFS to
	ctf_dump_format_type. Don't trim off excess linefeeds now we no
	longer generate them.  Dump enumerated types.

C allows you to do only a very few things with entities of incomplete
type (as opposed to pointers to them): make pointers to them and give
them cv-quals, roughly. In particular you can't sizeof them and you
can't get their alignment.

We cannot impose all the requirements the standard imposes on CTF users,
because the deduplicator can transform any structure type into a forward
for the purposes of breaking cycles: so CTF type graphs can easily
contain things like arrays of forward type (if you want to figure out
their size or alignment, you need to chase down the types this forward
might be a forward to in child TU dicts: we will soon add API functions
to make doing this much easier).

Nonetheless, it is still meaningless to ask for the size or alignment of
forwards: but libctf didn't prohibit this and returned nonsense from
internal implementation details when you asked (it returned the kind of
the pointed-to type as both the size and alignment, because forwards
reuse ctt_type as a type kind, and ctt_type and ctt_size overlap).  So
introduce a new error, ECTF_INCOMPLETE, which is returned when you try
to get the size or alignment of forwards: we also return it when you try
to do things that require libctf itself to get the size or alignment of
a forward, notably using a forward as an array index type (which C
should never do in any case) or adding forwards to structures without
specifying their offset explicitly.

The dumper will not emit size or alignment info for forwards any more.

(This should not be an API break since ctf_type_size and ctf_type_align
could both return errors before now: any code that isn't expecting error
returns is already potentially broken.)

include/ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-api.h (ECTF_INCOMPLETE): New.
	(ECTF_NERR): Adjust.

ld/ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* testsuite/ld-ctf/conflicting-cycle-1.parent.d: Adjust for dumper
	changes.
	* testsuite/ld-ctf/cross-tu-cyclic-conflicting.d: Likewise.
	* testsuite/ld-ctf/forward.c: New test...
	* testsuite/ld-ctf/forward.d: ... and results.

libctf/ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-types.c (ctf_type_resolve): Improve comment.
	(ctf_type_size): Yield ECTF_INCOMPLETE when applied to forwards.
	Emit errors into the right dict.
	(ctf_type_align): Likewise.
	* ctf-create.c (ctf_add_member_offset): Yield ECTF_INCOMPLETE
	when adding a member without explicit offset when this member, or
	the previous member, is incomplete.
	* ctf-dump.c (ctf_dump_format_type): Do not try to print the size of
	forwards.
	(ctf_dump_member): Do not try to print their alignment.

Dump more details about the types found in data object and function info
sections (the type ID and recursive info on the type itself, but not on
its members).  Before now, this was being dumped for entries in the
variable section, but not for the closely-related function info and data
object sections, which is inconsistent and makes finding the
corresponding types in the type section unnecessarily hard.  (This also
gets rid of code in which bugs have already been found in favour of the
same code everything else in the dumper uses to dump types.)

While we're doing that, change the recursive type dumper in question to
recursively dump info on arrays' element type, just as we do for all
types that reference other types. (Arrays are not a kind of reference
type in libctf, but perhaps we should change that in future and make
ctf_type_reference return the element type.)

ld/ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* testsuite/ld-ctf/array.d: Adjust for dumper changes.
	* testsuite/ld-ctf/data-func-conflicted.d: Likewise.
	* testsuite/ld-ctf/diag-cttname-null.d: Likewise.
	* testsuite/ld-ctf/diag-cuname.d: Likewise.
	* testsuite/ld-ctf/diag-parlabel.d: Likewise.
	* testsuite/ld-ctf/function.d: Likewise.
	* testsuite/ld-ctf/slice.d: Likewise.

libctf/ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-dump.c (ctf_dump_objts): Dump by calling ctf_dump_format_type.
	(ctf_dump_format_type): Don't emit the size for function objects.
	Dump the element type of arrays like we dump the pointed-to type of
	pointers, etc.

In most places in CTF dumper output, we emit 0x... for hex strings, but
in three places (top-level type IDs, string table offsets, and the file
magic number) we don't emit the 0x.

This is very confusing if by chance there are no hex digits in the
output.  Add 0x consistently to everything, and adjust tests
accordingly.  While we're at it, improve the indentation of the output
so that subsequent lines in aggregate output are indented by at least as
many columns as the colon in the type output.  (Subsequent indentation
is still 4 spaces at a time.)

ld/ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* testsuite/ld-ctf/array.d: Adjust for dumper changes.
	* testsuite/ld-ctf/conflicting-cycle-1.B-1.d: Likewise.
	* testsuite/ld-ctf/conflicting-cycle-1.B-2.d: Likewise.
	* testsuite/ld-ctf/conflicting-cycle-1.parent.d: Likewise.
	* testsuite/ld-ctf/conflicting-cycle-2.A-1.d: Likewise.
	* testsuite/ld-ctf/conflicting-cycle-2.A-2.d: Likewise.
	* testsuite/ld-ctf/conflicting-cycle-2.parent.d: Likewise.
	* testsuite/ld-ctf/conflicting-cycle-3.C-1.d: Likewise.
	* testsuite/ld-ctf/conflicting-cycle-3.C-2.d: Likewise.
	* testsuite/ld-ctf/conflicting-cycle-3.parent.d: Likewise.
	* testsuite/ld-ctf/conflicting-enums.d: Likewise.
	* testsuite/ld-ctf/conflicting-typedefs.d: Likewise.
	* testsuite/ld-ctf/cross-tu-cyclic-conflicting.d: Likewise.
	* testsuite/ld-ctf/cross-tu-cyclic-nonconflicting.d: Likewise.
	* testsuite/ld-ctf/cross-tu-into-cycle.d: Likewise.
	* testsuite/ld-ctf/cross-tu-noncyclic.d: Likewise.
	* testsuite/ld-ctf/cycle-1.d: Likewise.
	* testsuite/ld-ctf/cycle-2.A.d: Likewise.
	* testsuite/ld-ctf/cycle-2.B.d: Likewise.
	* testsuite/ld-ctf/cycle-2.C.d: Likewise.
	* testsuite/ld-ctf/data-func-conflicted.d: Likewise.
	* testsuite/ld-ctf/diag-cttname-null.d: Likewise.
	* testsuite/ld-ctf/diag-cuname.d: Likewise.
	* testsuite/ld-ctf/diag-parlabel.d: Likewise.
	* testsuite/ld-ctf/diag-wrong-magic-number-mixed.d: Likewise.
	* testsuite/ld-ctf/function.d: Likewise.
	* testsuite/ld-ctf/slice.d: Likewise.
	* testsuite/ld-ctf/super-sub-cycles.d: Likewise.

libctf/ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-dump.c (ctf_dump_format_type): Add 0x to hex type IDs.
	(ctf_dump_header): Add 0x to the hex magic number.
	(ctf_dump_str): Add 0x to the hex string offsets.
	(ctf_dump_membstate_t) <cdm_toplevel_indent>: New.
	(ctf_dump_type): Adjust.  Free it when we're done.
	(type_hex_digits): New.
	(ctf_dump_member): Align output depending on the width of the type
	ID being generated.  Use printf padding, not a loop, to generate
	indentation.

The CTF declarator stack code (used by ctf_type_aname() and thus
ultimately by ctf-dump.c and objdump --ctf etc) contains careful
code to prepend array declarators to the stack it's building up
on the grounds that array declarators are ordered inside out: only
they're not, they're ordered outside in.

This has led to our (non-upstreamed) compiler emitting array declarators
backwards for years, because it looks backwards in the dumper unless
it's actually emitted backwards into the CTF so the dumper can wrongly
reverse it again: but

  int[5][6]

should be an array of 6 int[5]s, not an array of 5 int[6]'s, so even if
the dumper gets it right, actual users calling ctf_array_info are going
to see a completely wrong type graph with the wrong bounds in it.

Fix trivial.

libctf/ChangeLog
2021-01-05  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-decl.c (ctf_decl_push): Don't print array decls backwards.

tui_win_info::refresh_window simply calls wrefresh, which internally
does a doupdate.
This redraws the source background window without the source pad.
Then prefresh of the source pad draws the actual source code on top,
which flickers.

By changing this to wnoutrefresh, the actual drawing on the screen is
only done once in the following prefresh, without flickering.

gdb/ChangeLog:

2021-01-05  Hannes Domani  <ssbssa@yahoo.de>

	* tui/tui-winsource.c (tui_source_window_base::refresh_window):
	Call wnoutrefresh instead of tui_win_info::refresh_window.

There a 2 spaces between the numbers and source code, but only one of
them was redrawn.
So if you increase the source window height, the second space keeps the
character of the border rectangle.

With this both spaces are redrawn, so the border rectangle character is
overwritten.

gdb/ChangeLog:

2021-01-05  Hannes Domani  <ssbssa@yahoo.de>

	* tui/tui-source.c (tui_source_window::show_line_number):
	Redraw second space after line number.

The smaxrow and smaxcol parameters of prefresh are the bottom right corner
of the text area inclusive, not exclusive.

And if the source window grows bigger in height, the pad has to grow as
well.

gdb/ChangeLog:

2021-01-05  Hannes Domani  <ssbssa@yahoo.de>

	PR tui/26927
	* tui/tui-winsource.c (tui_source_window_base::refresh_window):
	Fix source pad size in prefresh.
	(tui_source_window_base::show_source_content): Grow source pad
	if necessary.

	* testsuite/ld-scripts/sysroot-prefix.exp: Exclude some targets.

--exclude-libs makes symbols hidden, but that doesn't prevent them
being made dynamic for is_relocatable_executable targets.  Fix that.

	* elflink.c (bfd_elf_link_record_dynamic_symbol): Handle no_export
	for relocatable executable.

	* config.guess: Import from upstream.
	* config.sub: Likewise.

	PR ld/26256
	* testsuite/ld-elf/pr26256-1b.d: xfail s12z.
	* testsuite/ld-scripts/crossref.exp (cross1): Don't xfail ia64.

	* vms-alpha.c (_bfd_vms_slurp_egsd): Read flags after size check.

When the ifunc resolver is in the executable, we may relax the variables
to gp-relative access instruction in the ifunc resolver, or in other functions
that called by the ifunc resolver.  But this will cause the uninitialized
gp problem since the ifunc need to be resolved at the early runtime, that
is at the pre-load stage, but we set the gp until the startup code.

At first, we try to add a new dynamic tag, DT_RISCV_GP, to stroe the gp value
and let ld.so can init the gp register early, before the pre-load stage.  But
we need to extend the ABI if we want to add a new dynamic tag.  Therefore,
in the psabi discussion, we try another solution, which was suggested by the
lld and FreeBSD linker experts, to let ld.so set the gp earlier - make sure
__global_pointer$ is output as a dynamic symbol when we are generating pde,
since we only do the relaxation for it.  Afterwards, ld.so can search the
DT_SYMTAB to get the gp value, and set the gp register before resolving ifunc.

bfd/
    * elfnn-riscv.c (allocate_dynrelocs): When we are generating pde, make
      sure gp symbol is output as a dynamic symbol.

Make sure the files using atoi() include stdlib.h for its prototype.
These files were relying on it being included implicitly by others
which isn't guaranteed, and newer toolchains produce warnings.

Make sure the files using abs() include stdlib.h for its prototype.
These files were relying on it being included implicitly by others
which isn't guaranteed, and newer toolchains produce warnings.

This mirrors gdb behavior of dumping extra info when being run in
interactive mode.  It also gives us an excuse to use the otherwise
unused sim_print_config.

Meant to push this variant where naming preference is given to the
module the code resides in rather than the operation it performs.

This mirrors the existing sim_print_help function, and the behavior
of all other GNU tools with their --version.

print_sim_config has never been used anywhere, so rename it to follow
the sim_* naming style for all other symbols we export.

For non-relocatable link with SHF_LINK_ORDER inputs, allow mixed indirect
and data inputs with ordered and unordered inputs:

1. Add pattern to bfd_section for the matching section name pattern in
linker script and update BFD_FAKE_SECTION.
2. Sort the consecutive bfd_indirect_link_order sections with the same
pattern to allow linker script to overdide input section order.
3. Place unordered sections before ordered sections.
4. Change the offsets of the indirect input sections only.

bfd/

	PR ld/26256
	* elflink.c (compare_link_order): Place unordered sections before
	ordered sections.
	(elf_fixup_link_order): Add a link info argument.  Allow mixed
	ordered and unordered input sections for non-relocatable link.
	Sort the consecutive bfd_indirect_link_order sections with the
	same pattern.  Change the offsets of the bfd_indirect_link_order
	sections only.
	(bfd_elf_final_link): Pass info to elf_fixup_link_order.
	* section.c (bfd_section): Add pattern.
	(BFD_FAKE_SECTION): Initialize pattern to NULL.
	* bfd-in2.h: Regenerated.

gas/

	PR ld/26256
	* config/obj-elf.c (obj_elf_change_section): Also filter out
	SHF_LINK_ORDER.

ld/

	PR ld/26256
	* ldlang.c (gc_section_callback): Set pattern.
	* testsuite/ld-elf/pr26256-1.s: New file.
	* testsuite/ld-elf/pr26256-1.t: Likewise.
	* testsuite/ld-elf/pr26256-1a.d: Likewise.
	* testsuite/ld-elf/pr26256-1b.d: Likewise.
	* testsuite/ld-elf/pr26256-2.s: Likewise.
	* testsuite/ld-elf/pr26256-2.t: Likewise.
	* testsuite/ld-elf/pr26256-2a.d: Likewise.
	* testsuite/ld-elf/pr26256-2b-alt.d: Likewise.
	* testsuite/ld-elf/pr26256-2b.d: Likewise.
	* testsuite/ld-elf/pr26256-3.s: Likewise.
	* testsuite/ld-elf/pr26256-3a.d: Likewise.
	* testsuite/ld-elf/pr26256-3a.t: Likewise.
	* testsuite/ld-elf/pr26256-3b.d: Likewise.
	* testsuite/ld-elf/pr26256-3b.t: Likewise.

While working on PR26935 I noticed that when running test-case
gdb.base/morestack.exp with target board unix/-m32/-fPIE/-pie and ld linker,
I get this linetable fragment for morestack.S using readelf -wL:
...
CU: ../../../../libgcc/config/i386/morestack.S:
Line number    Starting address    View    Stmt
109               0xc9c               x
  ...
838               0xe03               x
  -               0xe04

636                   0               x
637                 0x3               x
  -                 0x4
...
but with "maint info line-table" I get:
...
INDEX  LINE   ADDRESS            IS-STMT
0      END    0x00000004         Y
1      109    0x00000c9c         Y
  ...
110    838    0x00000e03         Y
111    END    0x00000e04         Y
...

So, apparently the entries with addresses 0x0 and 0x3 are filtered out
because the addresses are out of range, but the same doesn't happen with the
end-of-seq terminator.

Fix this by filtering out end-of-seq terminators that do not actually
terminate anything.

Tested on x86_64-linux.

gdb/ChangeLog:

2021-01-04  Tom de Vries  <tdevries@suse.de>

	* buildsym.c (buildsym_compunit::record_line): Filter out end-of-seq
	terminators that do not terminate anything.

gdb/testsuite/ChangeLog:

2021-01-04  Tom de Vries  <tdevries@suse.de>

	* gdb.dwarf2/dw2-out-of-range-end-of-seq.exp: New file.

I spent a lot of time reading infrun debug logs recently, and I think
they could be made much more readable by being indented, to clearly see
what operation is done as part of what other operation.  In the current
format, there are no visual cues to tell where things start and end,
it's just a big flat list.  It's also difficult to understand what
caused a given operation (e.g. a call to resume_1) to be done.

To help with this, I propose to add the new scoped_debug_start_end
structure, along with a bunch of macros to make it convenient to use.

The idea of scoped_debug_start_end is simply to print a start and end
message at construction and destruction.  It also increments/decrements
a depth counter in order to make debug statements printed during this
range use some indentation.  Some care is taken to handle the fact that
debug can be turned on or off in the middle of such a range.  For
example, a "set debug foo 1" command in a breakpoint command, or a
superior GDB manually changing the debug_foo variable.

Two macros are added in gdbsupport/common-debug.h, which are helpers to
define module-specific macros:

  - scoped_debug_start_end: takes a message that is printed both at
    construction / destruction, with "start: " and "end: " prefixes.
  - scoped_debug_enter_exit: prints hard-coded "enter" and "exit"
    messages, to denote the entry and exit of a function.

I added some examples in the infrun module to give an idea of how it can
be used and what the result looks like.  The macros are in capital
letters (INFRUN_SCOPED_DEBUG_START_END and
INFRUN_SCOPED_DEBUG_ENTER_EXIT) to mimic the existing SCOPE_EXIT, but
that can be changed if you prefer something else.

Here's an excerpt of the debug
statements printed when doing "continue", where a displaced step is
started:

    [infrun] proceed: enter
      [infrun] proceed: addr=0xffffffffffffffff, signal=GDB_SIGNAL_DEFAULT
      [infrun] global_thread_step_over_chain_enqueue: enqueueing thread Thread 0x7ffff75a5640 (LWP 2289301) in global step over chain
      [infrun] start_step_over: enter
        [infrun] start_step_over: stealing global queue of threads to step, length = 1
        [infrun] start_step_over: resuming [Thread 0x7ffff75a5640 (LWP 2289301)] for step-over
        [infrun] resume_1: step=1, signal=GDB_SIGNAL_0, trap_expected=1, current thread [Thread 0x7ffff75a5640 (LWP 2289301)] at 0x5555555551bd
        [displaced] displaced_step_prepare_throw: displaced-stepping Thread 0x7ffff75a5640 (LWP 2289301) now
        [displaced] prepare: selected buffer at 0x5555555550c2
        [displaced] prepare: saved 0x5555555550c2: 1e fa 31 ed 49 89 d1 5e 48 89 e2 48 83 e4 f0 50
        [displaced] amd64_displaced_step_copy_insn: copy 0x5555555551bd->0x5555555550c2: c7 45 fc 00 00 00 00 eb 13 8b 05 d4 2e 00 00 83
        [displaced] displaced_step_prepare_throw: prepared successfully thread=Thread 0x7ffff75a5640 (LWP 2289301), original_pc=0x5555555551bd, displaced_pc=0x5555555550c2
        [displaced] resume_1: run 0x5555555550c2: c7 45 fc 00
        [infrun] infrun_async: enable=1
        [infrun] prepare_to_wait: prepare_to_wait
        [infrun] start_step_over: [Thread 0x7ffff75a5640 (LWP 2289301)] was resumed.
        [infrun] operator(): step-over queue now empty
      [infrun] start_step_over: exit
      [infrun] proceed: start: resuming threads, all-stop-on-top-of-non-stop
        [infrun] proceed: resuming Thread 0x7ffff7da7740 (LWP 2289296)
        [infrun] resume_1: step=0, signal=GDB_SIGNAL_0, trap_expected=0, current thread [Thread 0x7ffff7da7740 (LWP 2289296)] at 0x7ffff7f7d9b7
        [infrun] prepare_to_wait: prepare_to_wait
        [infrun] proceed: resuming Thread 0x7ffff7da6640 (LWP 2289300)
        [infrun] resume_1: thread Thread 0x7ffff7da6640 (LWP 2289300) has pending wait status status->kind = stopped, signal = GDB_SIGNAL_TRAP (currently_stepping=0).
        [infrun] prepare_to_wait: prepare_to_wait
        [infrun] proceed: [Thread 0x7ffff75a5640 (LWP 2289301)] resumed
        [infrun] proceed: resuming Thread 0x7ffff6da4640 (LWP 2289302)
        [infrun] resume_1: thread Thread 0x7ffff6da4640 (LWP 2289302) has pending wait status status->kind = stopped, signal = GDB_SIGNAL_TRAP (currently_stepping=0).
        [infrun] prepare_to_wait: prepare_to_wait
      [infrun] proceed: end: resuming threads, all-stop-on-top-of-non-stop
    [infrun] proceed: exit

We can easily see where the call to `proceed` starts and end.  We can
also see why there are a bunch of resume_1 calls, it's because we are
resuming threads, emulating all-stop on top of a non-stop target.

We also see that debug statements nest well with other modules that have
been migrated to use the "new" debug statement helpers (because they all
use debug_prefixed_vprintf in the end.  I think this is desirable, for
example we could see the debug statements about reading the DWARF info
of a library nested under the debug statements about loading that
library.

Of course, modules that haven't been migrated to use the "new" helpers
will still print without indentations.  This will be one good reason to
migrate them.

I think the runtime cost (when debug statements are disabled) of this is
reasonable, given the improvement in readability.  There is the cost of
the conditionals (like standard debug statements), one more condition
(if (m_must_decrement_print_depth)) and the cost of constructing a stack
object, which means copying a fews pointers.

Adding the print in fetch_inferior_event breaks some tests that use "set
debug infrun", because it prints a debug statement after the prompt.  I
adapted these tests to cope with it, by using the "-prompt" switch of
gdb_test_multiple to as if this debug statement is part of the expected
prompt.  It's unfortunate that we have to do this, but I think the debug
print is useful, and I don't want a few tests to get in the way of
adding good debug output.

gdbsupport/ChangeLog:

	* common-debug.h (debug_print_depth): New.
	(struct scoped_debug_start_end): New.
	(scoped_debug_start_end): New.
	(scoped_debug_enter_exit): New.
	* common-debug.cc (debug_prefixed_vprintf): Print indentation.

gdb/ChangeLog:

	* debug.c (debug_print_depth): New.
	* infrun.h (INFRUN_SCOPED_DEBUG_START_END): New.
	(INFRUN_SCOPED_DEBUG_ENTER_EXIT): New.
	* infrun.c (start_step_over): Use
	INFRUN_SCOPED_DEBUG_ENTER_EXIT.
	(proceed): Use INFRUN_SCOPED_DEBUG_ENTER_EXIT and
	INFRUN_SCOPED_DEBUG_START_END.
	(fetch_inferior_event): Use INFRUN_SCOPED_DEBUG_ENTER_EXIT.

gdbserver/ChangeLog:

	* debug.cc (debug_print_depth): New.

gdb/testsuite/ChangeLog:

        * gdb.base/ui-redirect.exp: Expect infrun debug print after
	prompt.
        * gdb.threads/ia64-sigill.exp: Likewise.
        * gdb.threads/watchthreads-reorder.exp: Likewise.

Change-Id: I7c3805e6487807aa63a1bae318876a0c69dce949

The code in print_target_wait_results uses a single call to debug_printf
in order to make sure a single timestamp is emitted, despite printing
multiple lines.  The result is:

    941502.043284 [infrun] target_wait (-1.0.0, status) =
    [infrun]   649832.649832.0 [process 649832],
    [infrun]   status->kind = stopped, signal = GDB_SIGNAL_TRAP

I find this decision a bit counter productive, because it messes up the
alignment of the three lines.  We don't care that three (slightly
different) timestamps are printed.

I suggest to change this function to use infrun_debug_printf, with this
result:

    941601.425771 [infrun] print_target_wait_results: target_wait (-1.0.0 [process -1], status) =
    941601.425824 [infrun] print_target_wait_results:   651481.651481.0 [process 651481],
    941601.425867 [infrun] print_target_wait_results:   status->kind = stopped, signal = GDB_SIGNAL_TRAP

Note that the current code only prints the waiton_ptid as a string
between square brackets if pid != -1.  I don't think this complexity is
needed in a debug print.  I made it so it's always printed, which I
think results in a much simpler function.

gdb/ChangeLog:

	* infrun.c (print_target_wait_results): Use infrun_debug_printf.

Change-Id: I817bd10286b8e641a6c751ac3a1bd1ddf9b18ce0