aboutsummaryrefslogtreecommitdiff
path: root/libctf/testsuite
AgeCommit message (Collapse)AuthorFilesLines
2024-09-24binutils testsuite: canonicalize subtest names in libctfFrank Ch. Eigler2-9/+8
Previous code included the full $srcdir pathnames in the individual subtest PASS/FAIL names, which makes it difficult to compute comparisons or regressions between test runs on different machines. This version switches to the basename only, which are common. Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
2024-07-31libctf, include: add ctf_dict_set_flag: less enum dup checking by defaultNick Alcock1-3/+64
The recent change to detect duplicate enum values and return ECTF_DUPLICATE when found turns out to perturb a great many callers. In particular, the pahole-created kernel BTF has the same problem we historically did, and gleefully emits duplicated enum constants in profusion. Handling the resulting duplicate errors from BTF -> CTF converters reasonably is unreasonably difficult (it amounts to forcing them to skip some types or reimplement the deduplicator). So let's step back a bit. What we care about mostly is that the deduplicator treat enums with conflicting enumeration constants as conflicting types: programs that want to look up enumeration constant -> value mappings using the new APIs to do so might well want the same checks to apply to any ctf_add_* operations they carry out (and since they're *using* the new APIs, added at the same time as this restriction was imposed, there is likely to be no negative consequence of this). So we want some way to allow processes that know about duplicate detection to opt into it, while allowing everyone else to stay clear of it: but we want ctf_link to get this behaviour even if its caller has opted out. So add a new concept to the API: dict-wide CTF flags, set via ctf_dict_set_flag, obtained via ctf_dict_get_flag. They are not bitflags but simple arbitrary integers and an on/off value, stored in an unspecified manner (the one current flag, we translate into an LCTF_* flag value in the internal ctf_dict ctf_flags word). If you pass in an invalid flag or value you get a new ECTF_BADFLAG error, so the caller can easily tell whether flags added in future are valid with a particular libctf or not. We check this flag in ctf_add_enumerator, and set it around the link (including on child per-CU dicts). The newish enumerator-iteration test is souped up to check the semantics of the flag as well. The fact that the flag can be set and unset at any time has curious consequences. You can unset the flag, insert a pile of duplicates, then set it and expect the new duplicates to be detected, not only by ctf_add_enumerator but also by ctf_lookup_enumerator. This means we now have to maintain the ctf_names and conflicting_enums enum-duplication tracking as new enums are added, not purely as the dict is opened. Move that code out of init_static_types_internal and into a new ctf_track_enumerator function that addition can also call. (None of this affects the file format or serialization machinery, which has to be able to handle duplicate enumeration constants no matter what.) include/ * ctf-api.h (CTF_ERRORS) [ECTF_BADFLAG]: New. (ECTF_NERR): Update. (CTF_STRICT_NO_DUP_ENUMERATORS): New flag. (ctf_dict_set_flag): New function. (ctf_dict_get_flag): Likewise. libctf/ * ctf-impl.h (LCTF_STRICT_NO_DUP_ENUMERATORS): New flag. (ctf_track_enumerator): Declare. * ctf-dedup.c (ctf_dedup_emit_type): Set it. * ctf-link.c (ctf_create_per_cu): Likewise. (ctf_link_deduplicating_per_cu): Likewise. (ctf_link): Likewise. (ctf_link_write): Likewise. * ctf-subr.c (ctf_dict_set_flag): New function. (ctf_dict_get_flag): New function. * ctf-open.c (init_static_types_internal): Move enum tracking to... * ctf-create.c (ctf_track_enumerator): ... this new function. (ctf_add_enumerator): Call it. * libctf.ver: Add the new functions. * testsuite/libctf-lookup/enumerator-iteration.c: Test them.
2024-07-31libctf: fix CTF dict compressionNick Alcock2-0/+160
Commit 483546ce4f3 ("libctf: make ctf_serialize() actually serialize") accidentally broke dict compression. There were two bugs: - ctf_arc_write_one_ctf was still making its own decision about whether to compress the dict via direct ctf_size comparison, which is unfortunate because now that it no longer calls ctf_serialize itself, ctf_size is always zero when it does this: it should let the writing functions decide on the threshold, which they contain code to do which is simply not used for lack of one trivial wrapper to write to an fd and also provide a compression threshold - ctf_write_mem, the function underlying all writing as of the commit above, was calling zlib's compressBound and avoiding compression if this returned a value larger than the input. Unfortunately compressBound does not do a trial compression and determine whether the result is compressible: it just adds zlib header sizes to the value passed in, so our test would *always* have concluded that the value was incompressible! Avoid by simply always compressing if the raw size is larger than the threshold: zlib is quite clever enough to avoid actually compressing if the data is incompressible. Add a testcase for this. libctf/ * ctf-impl.h (ctf_write_thresholded): New... * ctf-serialize.c (ctf_write_thresholded): ... defined here, a wrapper around... (ctf_write_mem): ... this. Don't check compressibility. (ctf_compress_write): Reimplement as a ctf_write_thresholded wrapper. (ctf_write): Likewise. * ctf-archive.c (arc_write_one_ctf): Just call ctf_write_thresholded rather than trying to work out whether to compress. * testsuite/libctf-writable/ctf-compressed.*: New test.
2024-07-31libctf: fix linking of non-root-visible typesNick Alcock2-0/+128
If you deduplicate non-root-visible types, the resulting type should still be non-root-visible! We were promoting all such types to root-visible, and re-demoting them only if their names collided (which might happen on cu-mapped links if multiple compilation units with conflicting types are fused into one child dict). This "worked" before now, in that linking at least didn't fail (if you don't mind having your non-root flag value destroyed if you're adding non-root-visible types), but now that conflicting enumerators cause their containing enums to become conflicted (enums which might have *different names*), this caused the linker to crash when it hit two enumerators with conflicting values. Not testable in ld because cu-mapped links are not exposed to ld, but can be tested via direct creation of libraries and calls to ctf_link directly. (This also tests the ctf_dump non-root type printout, which before now was untested.) libctf/ * ctf-dedup.c (ctf_dedup_emit_type): Non-root-visible input types should be emitted as non-root-visible output types. * testsuite/libctf-writable/ctf-nonroot-linking.c: New test. * testsuite/libctf-writable/ctf-nonroot-linking.lk: New test.
2024-06-19libctf: fix testsuite bugs revealed by -WallNick Alcock4-13/+9
Most of these are harmless, but some of the type confusions and especially a missing ctf_strerror() on an error path were actual bugs that could have resulted in test failures crashing rather than printing an error message. libctf/ * testsuite/libctf-lookup/enumerator-iteration.c: Fix type confusion, signedness confusion and a missing ctf_errmsg(). * testsuite/libctf-regression/libctf-repeat-cu-main.c: Return 0 from the test function. * testsuite/libctf-regression/open-error-free.c: Fix signedness confusion. * testsuite/libctf-regression/zrewrite.c: Remove unused label.
2024-06-18libctf, include: new functions for looking up enumeratorsNick Alcock3-0/+191
Three new functions for looking up the enum type containing a given enumeration constant, and optionally that constant's value. The simplest, ctf_lookup_enumerator, looks up a root-visible enumerator by name in one dict: if the dict contains multiple such constants (which is possible for dicts created by older versions of the libctf deduplicator), ECTF_DUPLICATE is returned. The next simplest, ctf_lookup_enumerator_next, is an iterator which returns all enumerators with a given name in a given dict, whether root-visible or not. The most elaborate, ctf_arc_lookup_enumerator_next, finds all enumerators with a given name across all dicts in an entire CTF archive, whether root-visible or not, starting looking in the shared parent dict; opened dicts are cached (as with all other ctf_arc_*lookup functions) so that repeated use does not incur repeated opening costs. All three of these return enumerator values as int64_t: unfortunately, API compatibility concerns prevent us from doing the same with the other older enum-related functions, which all return enumerator constant values as ints. We may be forced to add symbol-versioning compatibility aliases that fix the other functions in due course, bumping the soname for platforms that do not support such things. ctf_arc_lookup_enumerator_next is implemented as a nested ctf_archive_next iterator, and inside that, a nested ctf_lookup_enumerator_next iterator within each dict. To aid in this, add support to ctf_next_t iterators for iterators that are implemented in terms of two simultaneous nested iterators at once. (It has always been possible for callers to use as many nested or semi-overlapping ctf_next_t iterators as they need, which is one of the advantages of this style over the _iter style that calls a function for each thing iterated over: the iterator change here permits *ctf_next_t iterators themselves* to be implemented by iterating using multiple other iterators as part of their internal operation, transparently to the caller.) Also add a testcase that tests all these functions (which is fairly easy because ctf_arc_lookup_enumerator_next is implemented in terms of ctf_lookup_enumerator_next) in addition to enumeration addition in ctf_open()ed dicts, ctf_add_enumerator duplicate enumerator addition, and conflicting enumerator constant deduplication. include/ * ctf-api.h (ctf_lookup_enumerator): New. (ctf_lookup_enumerator_next): Likewise. (ctf_arc_lookup_enumerator_next): Likewise. libctf/ * libctf.ver: Add them. * ctf-impl.h (ctf_next_t) <ctn_next_inner>: New. * ctf-util.c (ctf_next_copy): Copy it. (ctf_next_destroy): Destroy it. * ctf-lookup.c (ctf_lookup_enumerator): New. (ctf_lookup_enumerator_next): New. * ctf-archive.c (ctf_arc_lookup_enumerator_next): New. * testsuite/libctf-lookup/enumerator-iteration.*: New test. * testsuite/libctf-lookup/enum-ctf-2.c: New test CTF, used by the above.
2024-06-18libctf: suppress spurious failure of malloc-counting tests under valgrindNick Alcock2-0/+18
The libctf-regression/open-error-free.c test works by interposing malloc and counting mallocs and frees across libctf operations. This only works under suitably-interposable mallocs on systems supporting dlsym (RTLD_NEXT, ...), so its operation is restricted to glibc systems for now, but also it interacts badly with valgrind, which interposes malloc itself. Detect a running valgrind and skip the test. Add new facilities allowing libctf lookup tests to declare themselves unsupported, by printing "UNSUPPORTED: " and then some meaningful message instead of their normal output. libctf/ * configure.ac: Check for <valgrind/valgrind.h>. * config.h.in: Regenerate. * configure: Likewise. * testsuite/lib/ctf-lib.exp (run_lookup_test): Add support for UNSUPPORTED tests. * testsuite/libctf-regression/open-error-free.c: When running under valgrind, this test is unsupported.
2024-06-12PR 31882 libctf: test suite incorrect format specifiersA. Wilcox2-2/+2
2024-05-23libctf testsuite compilation failureAlan Modra1-1/+1
* testsuite/libctf-regression/open-error-free.c (main): Correct format length modifier.
2024-05-17libctf: fix leak of entire dict when dict opening failsNick Alcock2-0/+190
Ever since commit 1fa7a0c24e78e7f ("libctf: sort out potential refcount loops") ctf_dict_close has only freed anything if the refcount on entry to the function is precisely 1. >1 obviously just decrements the refcount, but the linker machinery can sometimes cause freeing to recurse from a dict to another dict and then back to the first dict again, so we interpret a refcount of 0 as an indication that this is a recursive call and we should just return, because a caller is already freeing this dict. Unfortunately there is one situation in which this is not true: the bad: codepath in ctf_bufopen entered when opening fails. Because the refcount is bumped only at the very end of ctf_bufopen, any failure causes ctf_dict_close to be entered with a refcount of zero, and it frees nothing and we leak the entire dict. The solution is to bump the refcount to 1 right before freeing... but this codepath is clearly delicate enough that we need to properly validate it, so we add a test that uses malloc interposition to count allocations and frees, creates a dict, writes it out, intentionally corrupts it (by setting a bunch of bytes after the header to a value high enough that it is definitely not a valid CTF type kind), then tries to open it again and counts the malloc/free pairs to make sure they're matched. (Test run only on *-linux-gnu, because malloc interposition is not a thing you can rely upon working everywhere, and this test is not arch-dependent so if it passes on one arch it can be assumed to pass on all of them.) libctf/ * ctf-open.c (ctf_bufopen): Bump the refcount on failure. * testsuite/libctf-regression/open-error-free.*: New test.
2024-05-17libctf: test: add wrapperNick Alcock1-2/+11
This .lk option lets you run the lookup program via a wrapper executable. For example, to run under valgrind and check for leaks (albeit noisily because of the libtool shell script wrapper): libctf/ * testsuite/lib/ctf-lib.exp (run_lookup_test): Add wrapper.
2024-05-17libctf: test: add hostNick Alcock1-0/+9
This .lk option lets you execute particular tests only on specific host architectures. libctf/ * testsuite/lib/ctf-lib.exp (run_lookup_test): Add host.
2024-05-17libctf: test: add lookup_linkNick Alcock1-1/+5
This .lk option lets you link the lookup program with extra libraries in addition to -lctf. libctf/ * testsuite/lib/ctf-lib.exp (run_lookup_test): Add lookup_link.
2024-04-24buffer overflow in libctf testsAlan Modra2-2/+2
* testsuite/libctf-regression/gzrewrite.c (main): Don't overflow "a" buffer in "after adding types" check. * testsuite/libctf-regression/zrewrite.c (main): Likewise.
2024-04-20Error compiling libctf-regression testAlan Modra2-2/+2
Seen on 64-bit targets. ERROR: compilation of lookup program .../libctf-regression/gzrewrite.c failed * testsuite/libctf-regression/gzrewrite.c (main): Use %zu to print size_t values. * testsuite/libctf-regression/zrewrite.c (main): Likewise.
2024-04-19libctf: fix leak in testNick Alcock1-0/+4
This purely serves to make it easier to interpret valgrind output. No functional effect. libctf/ * testsuite/libctf-lookup/conflicting-type-syms.c: Free everything.
2024-04-19libctf: add rewriting testsNick Alcock5-0/+346
Now there's a chance of it actually working, we can add more tests for the long-broken dict read-and-rewrite cases. This is the first ever test for the (rarely-used, unpleasant, and until recently completely broken) ctf_gzwrite function. libctf/ * testsuite/libctf-regression/gzrewrite*: New test. * testsuite/libctf-regression/zrewrite*: Likewise.
2024-04-19libctf: make ctf_serialize() actually serializeNick Alcock1-5/+6
ctf_serialize() evolved from the old ctf_update(), which mutated the in-memory CTF dict to make all the dynamic in-memory types into static, unchanging written-to-the-dict types (by deserializing and reserializing it): back in the days when you could only do type lookups on static types, this meant you could see all the types you added recently, at the small, small cost of making it impossible to change those older types ever again and inducing an amortized O(n^2) cost if you actually wanted to add references to types you added at arbitrary times to later types. It also reset things so that ctf_discard() would throw away only types you added after the most recent ctf_update() call. Some time ago this was all changed so that you could look up dynamic types just as easily as static types: ctf_update() changed so that only its visible side-effect of affecting ctf_discard() remained: the old ctf_update() was renamed to ctf_serialize(), made internal to libctf, and called from the various functions that wrote files out. ... but it was still working by serializing and deserializing the entire dict, swapping out its guts with the newly-serialized copy in an invasive and horrible fashion that coupled ctf_serialize() to almost every field in the ctf_dict_t. This is totally useless, and fixing it is easy: just rip all that code out and have ctf_serialize return a serialized representation, and let everything use that directly. This simplifies most of its callers significantly. (It also points up another bug: ctf_gzwrite() failed to call ctf_serialize() at all, so it would only ever work for a dict you just ctf_write_mem()ed yourself, just for its invisible side-effect of serializing the dict!) This lets us simplify away a bunch of internal-only open-side functionality for overriding the syn_ext_strtab and some just-added functionality for forcing in an existing atoms table, without loss of functionality, and lets us lift the restriction on reserializing a dict that was ctf_open()ed rather than being ctf_create()d: it's now perfectly OK to open a dict, modify it (except for adding members to existing structs, unions, or enums, which fails with -ECTF_RDONLY), and write it out again, just as one would expect. libctf/ * ctf-serialize.c (ctf_symtypetab_sect_sizes): Fix typos. (ctf_type_sect_size): Add static type sizes too. (ctf_serialize): Return the new dict rather than updating the existing dict. No longer fail for dicts with static types; copy them onto the start of the new types table. (ctf_gzwrite): Actually serialize before gzwriting. (ctf_write_mem): Improve forced (test-mode) endian-flipping: flip dicts even if they are too small to be compressed. Improve confusing variable naming. * ctf-archive.c (arc_write_one_ctf): Don't bother to call ctf_serialize: both the functions we call do so. * ctf-string.c (ctf_str_create_atoms): Drop serializing case (atoms arg). * ctf-open.c (ctf_simple_open): Call ctf_bufopen directly. (ctf_simple_open_internal): Delete. (ctf_bufopen_internal): Delete/rename to ctf_bufopen: no longer bother with syn_ext_strtab or forced atoms table, serialization no longer needs them. * ctf-create.c (ctf_create): Call ctf_bufopen directly. * ctf-impl.h (ctf_str_create_atoms): Drop atoms arg. (ctf_simple_open_internal): Delete. (ctf_bufopen_internal): Likewise. (ctf_serialize): Adjust. * testsuite/libctf-lookup/add-to-opened.c: Adjust now that this is supposed to work.
2024-04-19libctf: support addition of types to dicts read via ctf_open()Nick Alcock3-0/+169
libctf has long declared deserialized dictionaries (out of files or ELF sections or memory buffers or whatever) to be read-only: back in the furthest prehistory this was not the case, in that you could add a few sorts of type to such dicts, but attempting to do so often caused horrible memory corruption, so I banned the lot. But it turns out real consumers want it (notably DTrace, which synthesises pointers to types that don't have them and adds them to the ctf_open()ed dicts if it needs them). Let's bring it back again, but without the memory corruption and without the massive code duplication required in days of yore to distinguish between static and dynamic types: the representation of both types has been identical for a few years, with the only difference being that types as a whole are stored in a big buffer for types read in via ctf_open and per-type hashtables for newly-added types. So we discard the internally-visible concept of "readonly dictionaries" in favour of declaring the *range of types* that were already present when the dict was read in to be read-only: you can't modify them (say, by adding members to them if they're structs, or calling ctf_set_array on them), but you can add more types and point to them. (The API remains the same, with calls sometimes returning ECTF_RDONLY, but now they do so less often.) This is a fairly invasive change, mostly because code written since the ban was introduced didn't take the possibility of a static/dynamic split into account. Some of these irregularities were hard to define as anything but bugs. Notably: - The symbol handling was assuming that symbols only needed to be looked for in dynamic hashtabs or static linker-laid-out indexed/ nonindexed layouts, but now we want to check both in case people added more symbols to a dict they opened. - The code that handles type additions wasn't checking to see if types with the same name existed *at all* (so you could do ctf_add_typedef (fp, "foo", bar) repeatedly without error). This seems reasonable for types you just added, but we probably *do* want to ban addition of types with names that override names we already used in the ctf_open()ed portion, since that would probably corrupt existing type relationships. (Doing things this way also avoids causing new errors for any existing code that was doing this sort of thing.) - ctf_lookup_variable entirely failed to work for variables just added by ctf_add_variable: you had to write the dict out and read it back in again before they appeared. - The symbol handling remembered what symbols you looked up but didn't remember their types, so you could look up an object symbol and then find it popping up when you asked for function symbols, which seems less than ideal. Since we had to rejig things enough to be able to distinguish function and object symbols internally anyway (in order to give suitable errors if you try to add a symbol with a name that already existed in the ctf_open()ed dict), this bug suddenly became more visible and was easily fixed. We do not (yet) support writing out dicts that have been previously read in via ctf_open() or other deserializer (you can look things up in them, but not write them out a second time). This never worked, so there is no incompatibility; if it is needed at a later date, the serializer is a little bit closer to having it work now (the only table we don't deal with is the types table, and that's because the upcoming CTFv4 changes are likely to make major changes to the way that table is represented internally, so adding more code that depends on its current form seems like a bad idea). There is a new testcase that tests much of this, in particular that modification of existing types is still banned and that you can add new ones and chase them without error. libctf/ * ctf-impl.h (struct ctf_dict.ctf_symhash): Split into... (ctf_dict.ctf_symhash_func): ... this and... (ctf_dict.ctf_symhash_objt): ... this. (ctf_dict.ctf_stypes): New, counts static types. (LCTF_INDEX_TO_TYPEPTR): Use it instead of CTF_RDWR. (LCTF_RDWR): Deleted. (LCTF_DIRTY): Renumbered. (LCTF_LINKING): Likewise. (ctf_lookup_variable_here): New. (ctf_lookup_by_sym_or_name): Likewise. (ctf_symbol_next_static): Likewise. (ctf_add_variable_forced): Likewise. (ctf_add_funcobjt_sym_forced): Likewise. (ctf_simple_open_internal): Adjust. (ctf_bufopen_internal): Likewise. * ctf-create.c (ctf_grow_ptrtab): Adjust a lot to start with. (ctf_create): Migrate a bunch of initializations into bufopen. Force recreation of name tables. Do not forcibly override the model, let ctf_bufopen do it. (ctf_static_type): New. (ctf_update): Drop LCTF_RDWR check. (ctf_dynamic_type): Likewise. (ctf_add_function): Likewise. (ctf_add_type_internal): Likewise. (ctf_rollback): Check ctf_stypes, not LCTF_RDWR. (ctf_set_array): Likewise. (ctf_add_struct_sized): Likewise. (ctf_add_union_sized): Likewise. (ctf_add_enum): Likewise. (ctf_add_enumerator): Likewise (only on the target dict). (ctf_add_member_offset): Likewise. (ctf_add_generic): Drop LCTF_RDWR check. Ban addition of types with colliding names. (ctf_add_forward): Note safety under the new rules. (ctf_add_variable): Split all but the existence check into... (ctf_add_variable_forced): ... this new function. (ctf_add_funcobjt_sym): Likewise... (ctf_add_funcobjt_sym_forced): ... for this new function. * ctf-link.c (ctf_link_add_linker_symbol): Ban calling on dicts with any stypes. (ctf_link_add_strtab): Likewise. (ctf_link_shuffle_syms): Likewise. (ctf_link_intern_extern_string): Note pre-existing prohibition. * ctf-lookup.c (ctf_lookup_by_id): Drop LCTF_RDWR check. (ctf_lookup_variable): Split out looking in a dict but not its parent into... (ctf_lookup_variable_here): ... this new function. (ctf_lookup_symbol_idx): Track whether looking up a function or object: cache them separately. (ctf_symbol_next): Split out looking in non-dynamic symtypetab entries to... (ctf_symbol_next_static): ... this new function. Don't get confused by the simultaneous presence of static and dynamic symtypetab entries. (ctf_try_lookup_indexed): Don't waste time looking up symbols by index before there can be any idea how symbols are numbered. (ctf_lookup_by_sym_or_name): Distinguish between function and data object lookups. Drop LCTF_RDWR. (ctf_lookup_by_symbol): Adjust. (ctf_lookup_by_symbol_name): Likewise. * ctf-open.c (init_types): Rename to... (init_static_types): ... this. Drop LCTF_RDWR. Populate ctf_stypes. (ctf_simple_open): Drop writable arg. (ctf_simple_open_internal): Likewise. (ctf_bufopen): Likewise. (ctf_bufopen_internal): Populate fields only used for writable dicts. Drop LCTF_RDWR. (ctf_dict_close): Cater for symhash cache split. * ctf-serialize.c (ctf_serialize): Use ctf_stypes, not LCTF_RDWR. * ctf-types.c (ctf_variable_next): Drop LCTF_RDWR. * testsuite/libctf-lookup/add-to-opened*: New test.
2024-04-19libctf: fix name lookup in dicts containing base-type bitfieldsNick Alcock2-0/+138
The intent of the name lookup code was for lookups to yield non-bitfield basic types except if none existed with a given name, and only then return bitfield types with that name. Unfortunately, the code as written only does this if the base type has a type ID higher than all bitfield types, which is most unlikely (the opposite is almost always the case). Adjust it so that what ends up in the name table is the highest-width zero-offset type with a given name, if any such exist, and failing that the first type with that name we see, no matter its offset. (We don't define *which* bitfield type you get, after all, so we might as well just stuff in the first we find.) Reported by Stephen Brennan <stephen.brennan@oracle.com>. libctf/ * ctf-open.c (init_types): Modify to allow some lookups during open; detect bitfield name reuse and prefer less bitfieldy types. * testsuite/libctf-writable/libctf-bitfield-name-lookup.*: New test.
2024-03-11libctf: fix uninitialized variables in testsuiteNick Alcock1-1/+1
Just because a path is an error path doesn't mean the program terminates there if you don't ask it to. And we don't want to -- but that means we need to initialize the variables that are missed if an error happens to *something*. Type ID 0 (unimplemented) will do: it'll induce further ECTF_BADID errors, but that's no bad thing. libctf/ChangeLog: * testsuite/libctf-writable/libctf-errors.c: Initialize variables.
2024-01-04Update year range in copyright notice of binutils filesAlan Modra6-6/+6
Adds two new external authors to etc/update-copyright.py to cover bfd/ax_tls.m4, and adds gprofng to dirs handled automatically, then updates copyright messages as follows: 1) Update cgen/utils.scm emitted copyrights. 2) Run "etc/update-copyright.py --this-year" with an extra external author I haven't committed, 'Kalray SA.', to cover gas testsuite files (which should have their copyright message removed). 3) Build with --enable-maintainer-mode --enable-cgen-maint=yes. 4) Check out */po/*.pot which we don't update frequently.
2023-10-20libctf: fix creation-time parent/child dict confusionsNick Alcock15-0/+263
The fixes applied a few years ago to resolve confusions between parent and child dicts at lookup time also apply in various forms to creation. In general, if you have a type in a parent dict ctf_imported into a child and you do something to it, and the parent dict is writable (created via ctf_create, not opened via ctf_open*) it should work just the same to make changes to that type via a child dict as it does to make the change to the parent dict directly -- and nothing you're prohibited from doing to the parent dict when done directly should be allowed just because you're doing it via a child. Specifically, the following don't work when doing things from the child, but should: - adding a member of a type in the parent to a struct or union in the parent via ctf_add_member or ctf_add_member_offset: this yields ECTF_BADID - adding a member of a type in the parent to a struct or union in the parent via ctf_add_member_encoded: this dumps core (!). - adding an enumerand to an enumerator in the parent: this yields ECTF_BADID - setting the properties of an array in the parent via ctf_set_array; this yields ECTF_BADID Relatedly, some things work when doing things via a child that should fail, yielding a CTF dictionary with invalid content (readable, but meaningless): in particular, you can add a child type to a struct in the parent via any of the ctf_add_member* family and nothing complains at all, even though you should never be able to add references to children to parents (since any given parent can be associated with many different children). A family of tests is added to check each of these cases independently, since some can result in coredumps and it would be nice to test the other cases even if some dump core. They use a common library to do all the actual work. The set of affected API calls was determined by code inspection (auditing all calls to ctf_dtd_lookup): it's possible that I missed a few, but I doubt it, since other cases use ctf_lookup* functions, which already climb to the parent where appropriate. libctf/ChangeLog: PR libctf/30985 * ctf-create.c (ctf_dtd_lookup): Traverse to parents if necessary. (ctf_set_array): Likewise. Report errors on the child; require both parent and child to be writable. (ctf_add_enumerator): Likewise. (ctf_add_member_offset): Likewise. Prohibit addition of child types to structs in the parent. (ctf_add_member_encoded): Do not dereference a NULL dtd: report ECTF_BADID instead. * ctf-string.c (ctf_str_add_ref_internal): Report ENOMEM on the dict if addition of a string ref fails. * testsuite/libctf-writable/parent-child-dtd-crash-lib.c: New library. * testsuite/libctf-writable/parent-child-dtd-enum.*: New test. * testsuite/libctf-writable/parent-child-dtd-enumerator.*: New test. * testsuite/libctf-writable/parent-child-dtd-member-encoded.*: New test. * testsuite/libctf-writable/parent-child-dtd-member-offset.*: New test. * testsuite/libctf-writable/parent-child-dtd-set-array.*: New test. * testsuite/libctf-writable/parent-child-dtd-struct.*: New test. * testsuite/libctf-writable/parent-child-dtd-union.*: New test.
2023-10-18libctf: check for problems with error returnsNick Alcock2-0/+75
We do this as a writable test because the only known-affected platforms (with ssize_t longer than unsigned long) use PE, and we do not have support for CTF linkage in the PE linker yet. PR libctf/30836 * libctf/testsuite/libctf-writable/libctf-errors.*: New test.
2023-04-08libctf: propagate errors from parents correctlyNick Alcock2-0/+165
CTF dicts have per-dict errno values: as with other errno values these are set on error and left unchanged on success. This means that all errors *must* set the CTF errno: if a call leaves it unchanged, the caller is apt to find a previous, lingering error and misinterpret it as the real error. There are many places in libctf where we carry out operations on parent dicts as a result of carrying out other user-requested operations on child dicts (e.g. looking up information on a pointer to a type will look up the type as well: the pointer might well be in a child and the type it's a pointer to in the parent). Those operations on the parent might fail; if they do, the error must be correctly reflected on the child that the user-visible operation was carried out on. In many places this was not happening. So, audit and fix all those places. Add tests for as many of those cases as possible so they don't regress. libctf/ * ctf-create.c (ctf_add_slice): Use the original dict. * ctf-lookup.c (ctf_lookup_variable): Propagate errors. (ctf_lookup_symbol_idx): Likewise. * ctf-types.c (ctf_member_next): Likewise. (ctf_type_resolve_unsliced): Likewise. (ctf_type_aname): Likewise. (ctf_member_info): Likewise. (ctf_type_rvisit): Likewise. (ctf_func_type_info): Set the error on the right dict. (ctf_type_encoding): Use the original dict. * testsuite/libctf-writable/error-propagation.*: New test.
2023-04-08libctf, tests: do not assume host and target have identical field offsetsNick Alcock2-1/+12
The newly-introduced libctf-lookup unnamed-field-info test checks C compiler-observed field offsets against libctf-computed ones by #including the testcase in the lookup runner as well as generating CTF for it. This only works if the host, on which the lookup runner is compiled and executed, is the same architecture as the target, for which the CTF is generated: when crossing, the trick may fail. So pass down an indication of whether this is a cross into the testsuite, and add a new no_cross flag to .lk files that is used to suppress test execution when a cross-compiler is being tested. libctf/ * Makefile.am (check_DEJAGNU): Pass down TEST_CROSS. * Makefile.in: Regenerated. * testsuite/lib/ctf-lib.exp (run_lookup_test): Use it to implement the new no_cross option. * testsuite/libctf-lookup/unnamed-field-info.lk: Mark as no_cross.
2023-03-24libctf: get the offsets of fields of unnamed structs/unions rightNick Alcock3-0/+117
We were failing to add the offsets of the containing struct/union in this case, leading to all offsets being relative to the unnamed struct/union itself. libctf/ PR libctf/30264 * ctf-types.c (ctf_member_info): Add the offset of the unnamed member of the current struct as necessary. * testsuite/libctf-lookup/unnamed-field-info*: New test.
2023-01-12libctf: skip the testsuite from inside dejagnuNick Alcock1-0/+5
The libctf testsuite uses Tcl try/catch to trap run_output errors. This is only supported in reasonably recent Tcls, so we detect the lack of try/catch and suppress the testsuite via an Automake conditional in its absence. But this turns out not to work: Automake produces a check-DEJAGNU target regardless of the value of this conditional and sticks it in an unconditionally-executed part of the makefile, so the testsuite gets executed anyway, and fails with a nasty-looking syntax error. We can't disable it by taking "dejagnu" out of AUTOMAKE_OPTIONS, because if you do that Automake stops you using RUNTEST, RUNTESTFLAGS and other variables users would expect to work. So move to disabling the testsuite from inside the testsuite itself, importing the value of the former Automake conditional as a Tcl variable and exiting very early in default.exp if it's false. * configure.ac (TCL_TRY): No longer an Automake conditional. Rename to... (HAVE_TCL_TRY): ... this. * Makefile.am: Drop TCL_TRY. (development.exp): Set have_tcl_try. * testsuite/config/default.exp: Exit if have_tcl_try is false. * configure: Regenerated. * Makefile.in: Likewise.
2023-01-01Update year range in copyright notice of binutils filesAlan Modra6-6/+6
The newer update-copyright.py fixes file encoding too, removing cr/lf on binutils/bfdtest2.c and ld/testsuite/ld-cygwin/exe-export.exp, and embedded cr in binutils/testsuite/binutils-all/ar.exp string match.
2022-06-21libctf: tests: prune warnings from compiler outputNick Alcock2-6/+6
We were failing to call prune_warnings appropriately, leading to false-positive test failures on some platforms (observed on sparclinux). libctf/ChangeLog: * testsuite/lib/ctf-lib.exp: Prune warnings from compiler and linker output. * testsuite/libctf-regression/libctf-repeat-cu.exp: Likewise, and ar output too.
2022-06-21libctf: fix linking together multiple objects derived from the same sourceNick Alcock5-0/+146
Right now, if you compile the same .c input repeatedly with CTF enabled and different compilation flags, then arrange to link all of these together, then things misbehave in various ways. libctf may conflate either inputs (if the .o files have the same name, say if they are stored in different .a archives), or per-CU outputs when conflicting types are found: the latter can lead to entirely spurious errors when it tries to produce multiple per-CU outputs with the same name (discarding all but the last, but then looking for types in the earlier ones which have just been thrown away). Fixing this is multi-pronged. Both inputs and outputs need to be differentiated in the hashtables libctf keeps them in: inputs with the same cuname and filename need to be considered distinct as long as they have different associated CTF dicts, and per-CU outputs need to be considered distinct as long as they have different associated input dicts. Right now there is nothing tying the two together other than the CU name: fix this by introducing a new field in the ctf_dict_t named ctf_link_in_out, which (for input dicts) points to the associated per-CU output dict (if any), and for output dicts points to the associated input dict. At creation time the name used is completely arbitrary: it's only important that it be distinct if CTF dicts are distinct. So, when a clash is found, adjust the CU name by sticking the number of elements in the input on the end. At output time, the CU name will appear in the linked object, so it matters a little more that it look slightly less ugly: in conflicting cases, append an incrementing integer, starting at 0. This naming scheme is not very helpful, but it's hard to see what else we can do. The input .o name may be the same. The input .a name is not even visible to ctf_link, and even *that* might be the same, because .a's can contain many members with the same name, all of which participate in the link. All we really know is that the two have distinct dictionaries with distinct types in them, and at least this way they are all represented, any any symbols, variables etc referring to those types are accurately stored. (As a side-effect this also fixes a use-after-free and double-free when errors are found during variable or symbol emission.) Use the opportunity to prevent a couple of sources of problems, to wit changing the active CU mappings when a link has already been done (no effect on ld, which doesn't use CU mappings at all), and causing multiple consecutive ctf_link's to have the same net effect as just doing the last one (no effect on ld, which only ever does one ctf_link) rather than having the links be a sort of half-incremental not-really-intended mess. libctf/ChangeLog: PR libctf/29242 * ctf-impl.h (struct ctf_dict) [ctf_link_in_out]: New. * ctf-dedup.c (ctf_dedup_emit_type): Set it. * ctf-link.c (ctf_link_add_ctf_internal): Set the input CU name uniquely when clashes are found. (ctf_link_add): Document what repeated additions do. (ctf_new_per_cu_name): New, come up with a consistent name for a new per-CU dict. (ctf_link_deduplicating): Use it. (ctf_create_per_cu): Use it, and ctf_link_in_out, and set ctf_link_in_out properly. Don't overwrite per-CU dicts with per-CU dicts relating to different inputs. (ctf_link_add_cu_mapping): Prevent per-CU mappings being set up if we already have per-CU outputs. (ctf_link_one_variable): Adjust ctf_link_per_cu call. (ctf_link_deduplicating_one_symtypetab): Likewise. (ctf_link_empty_outputs): New, delete all the ctf_link_outputs and blank out ctf_link_in_out on the corresponding inputs. (ctf_link): Clarify the effect of multiple ctf_link calls. Empty ctf_link_outputs if it already exists rather than having the old output leak into the new link. Fix a variable name. * testsuite/config/default.exp (AR): Add. (OBJDUMP): Likewise. * testsuite/libctf-regression/libctf-repeat-cu.exp: New test. * testsuite/libctf-regression/libctf-repeat-cu*: Main program, library, and expected results for the test.
2022-01-02Update year range in copyright notice of binutils filesAlan Modra5-5/+5
The result of running etc/update-copyright.py --this-year, fixing all the files whose mode is changed by the script, plus a build with --enable-maintainer-mode --enable-cgen-maint=yes, then checking out */po/*.pot which we don't update frequently. The copy of cgen was with commit d1dd5fcc38ead reverted as that commit breaks building of bfp opcodes files.
2021-09-27libctf, lookup: fix bounds of pptrtab lookupNick Alcock2-0/+70
An off-by-one bug in the check for pptrtab lookup meant that we could access the pptrtab past its bounds (*well* past its bounds), particularly if we called ctf_lookup_by_name in a child dict with "*foo" where "foo" is a type that exists in the parent but not the child and no previous lookups by name have been carried out. (Note that "*foo" is not even a valid thing to call ctf_lookup_by_name with: foo * is. Nonetheless, users sometimes do call ctf_lookup_by_name with invalid content, and it should return ECTF_NOTYPE, not crash.) ctf_pptrtab_len, as its name suggests (and as other tests of it in ctf-lookup.c confirm), is one higher than the maximum valid permissible index, so the comparison is wrong. (Test added, which should fail pretty reliably in the presence of this bug on any machine with 4KiB pages.) libctf/ChangeLog 2021-09-27 Nick Alcock <nick.alcock@oracle.com> * ctf-lookup.c (ctf_lookup_by_name_internal): Fix pptrtab bounds. * testsuite/libctf-writable/pptrtab-writable-page-deep-lookup.*: New test.
2021-09-27libctf, testsuite: fix various warnings in testsNick Alcock10-18/+10
These warnings are all off by default, but if they do fire you get spurious ERRORs when running make check-libctf. libctf/ChangeLog 2021-09-27 Nick Alcock <nick.alcock@oracle.com> * testsuite/libctf-lookup/enum-symbol.c: Remove unused label. * testsuite/libctf-lookup/conflicting-type-syms.c: Remove unused variables. * testsuite/libctf-regression/pptrtab.c: Likewise. * testsuite/libctf-regression/type-add-unnamed-struct.c: Likewise. * testsuite/libctf-writable/pptrtab.c: Likewise. * testsuite/libctf-writable/reserialize-strtab-corruption.c: Likewise. * testsuite/libctf-regression/nonstatic-var-section-ld-r.c: Fix format string. * testsuite/libctf-regression/nonstatic-var-section-ld.c: Likewise. * testsuite/libctf-regression/nonstatic-var-section-ld.lk: Adjust. * testsuite/libctf-writable/symtypetab-nonlinker-writeout.c: Fix initializer.
2021-09-03CC_FOR_TARGET et alAlan Modra2-11/+11
The top level Makefile, the ld Makefile and others, define CC_FOR_TARGET to be a compiler for the binutils target machine. This is the compiler that should be used for almost all tests with C source. There are _FOR_TARGET versions of CFLAGS, CXX, and CXXFLAGS too. This was all supposed to work with the testsuite .exp files using CC for the target compiler, and CC_FOR_HOST for the host compiler, with the makefiles passing CC=$CC_FOR_TARGET and CC_FOR_HOST=$CC to the runtest invocation. One exception to the rule of using CC_FOR_TARGET is the native-only ld bootstrap test, which uses the newly built ld to link a copy of itself. Since the files being linked were created with the host compiler, the boostrap test should use CC and CFLAGS, in case some host compiler option provides needed libraries automatically. However, bootstrap.exp used CC where it should have used CC_FOR_HOST. I set about fixing that problem, then decided that playing games in the makefiles with CC was a bad idea. Not only is it confusing, but other dejagnu code knows about CC_FOR_TARGET. See dejagnu/target.exp. So this patch gets rid of the makefile variable renaming and changes all the .exp files to use the correct _FOR_TARGET variables. CC_FOR_HOST and CFLAGS_FOR_HOST disappear. A followup patch will correct bootstrap.exp to use CFLAGS, and a number of other things I noticed. binutils/ * testsuite/lib/binutils-common.exp (run_dump_test): Use CC_FOR_TARGET and CFLAGS_FOR_TARGET rather than CC and CFLAGS. ld/ * Makefile.am (check-DEJAGNU): Don't set CC to CC_FOR_TARGET and similar. Pass variables with unchanged names. Don't set CC_FOR_HOST or CFLAGS_FOR_HOST. * Makefile.in: Regenerate. * testsuite/config/default.exp: Update default CC and similar. (compiler_supports, plug_opt): Use CC_FOR_TARGET. * testsuite/ld-cdtest/cdtest.exp: Replace all uses of CC with CC_FOR_TARGET, and similarly for CFLAGS, CXX and CXXFLAGS. * testsuite/ld-auto-import/auto-import.exp: Likewise. * testsuite/ld-cygwin/exe-export.exp: Likewise. * testsuite/ld-elf/dwarf.exp: Likewise. * testsuite/ld-elf/indirect.exp: Likewise. * testsuite/ld-elf/shared.exp: Likewise. * testsuite/ld-elfcomm/elfcomm.exp: Likewise. * testsuite/ld-elfvers/vers.exp: Likewise. * testsuite/ld-elfvsb/elfvsb.exp: Likewise. * testsuite/ld-elfweak/elfweak.exp: Likewise. * testsuite/ld-gc/gc.exp: Likewise. * testsuite/ld-ifunc/ifunc.exp: Likewise. * testsuite/ld-mn10300/mn10300.exp: Likewise. * testsuite/ld-pe/pe-compile.exp: Likewise. * testsuite/ld-pe/pe-run.exp: Likewise. * testsuite/ld-pe/pe-run2.exp: Likewise. * testsuite/ld-pie/pie.exp: Likewise. * testsuite/ld-plugin/lto.exp: Likewise. * testsuite/ld-plugin/plugin.exp: Likewise. * testsuite/ld-scripts/crossref.exp: Likewise. * testsuite/ld-selective/selective.exp: Likewise. * testsuite/ld-sh/sh.exp: Likewise. * testsuite/ld-shared/shared.exp: Likewise. * testsuite/ld-srec/srec.exp: Likewise. * testsuite/ld-undefined/undefined.exp: Likewise. * testsuite/ld-unique/unique.exp: Likewise. * testsuite/ld-x86-64/tls.exp: Likewise. * testsuite/lib/ld-lib.exp: Likewise. libctf/ * Makefile.am (check-DEJAGNU): Don't set CC to CC_FOR_TARGET. Pass CC and CC_FOR_TARGET. Don't set CC_FOR_HOST. * Makefile.in: Regenerate. * testsuite/config/default.exp: Update default CC and similar. * testsuite/lib/ctf-lib.exp (run_native_host_cmd): Use CC rather than CC_FOR_HOST. (run_lookup_test): Use CC_FOR_TARGET and CFLAGS_FOR_TARGET.
2021-05-06libctf, ld: fix test results for upstream GCCNick Alcock2-3/+2
The tests currently in binutils are aimed at the original GCC-based implementation of CTF, which emitted CTF directly from GCC's internal representation. The approach now under review emits CTF from DWARF, with an eye to eventually doing this for all non-DWARF debuginfo-like formats GCC supports. It also uses a different flag to enable CTF emission (-gctf rather than -gt). Adjust the testsuite accordingly. Given that the ld testsuite results are dependent on type ordering, which we do not guarantee at all, it's amazing how little changes. We see a few type ordering differences, slices change because the old GCC was buggy (slices were emitted "backwards", from the wrong end of the machine word) and its expected results were wrong, and GCC now emits the underlying integral type for enumerated types, though CTF has no way to record this yet (coming in v4). GCC also now emits even hidden symbols into the symtab (and thus symtypetab), so one symtypetab test changes its expected results slightly to compensate. Also add tests for the CTF_K_UNKNOWN nonrepresentable type: this couldn't be done before now since the only GCC that emits CTF_K_UNKNOWN for nonrepresentable types is the new one. ld/ChangeLog 2021-05-06 Nick Alcock <nick.alcock@oracle.com> * testsuite/ld-ctf/ctf.exp: Use -gctf, not -gt. * testsuite/lib/ld-lib.exp: Likewise. * testsuite/ld-ctf/nonrepresentable-1.c: New test for nonrepresentable types. * testsuite/ld-ctf/nonrepresentable-2.c: Likewise. * testsuite/ld-ctf/nonrepresentable.d: Likewise. * testsuite/ld-ctf/array.d: Larger type section. * testsuite/ld-ctf/data-func-conflicted.d: Likewise. * testsuite/ld-ctf/enums.d: Likewise. * testsuite/ld-ctf/conflicting-enums.d: Don't compare types. * testsuite/ld-ctf/cross-tu-cyclic-conflicting.d: Changed type order. * testsuite/ld-ctf/cross-tu-noncyclic.d: Likewise. * testsuite/ld-ctf/slice.d: Adjust for improved slice emission. libctf/ChangeLog 2021-05-06 Nick Alcock <nick.alcock@oracle.com> * testsuite/lib/ctf-lib.exp: Use -gctf, not -gt. * testsuite/libctf-regression/nonstatic-var-section-ld-r.lk: Hidden symbols now get into the symtypetab anyway.
2021-03-25libctf: fix memory leak in a testNick Alcock1-0/+1
Harmless, but causes noise that makes it harder to spot other leaks. libctf/ChangeLog 2021-03-25 Nick Alcock <nick.alcock@oracle.com> * testsuite/libctf-writable/symtypetab-nonlinker-writeout.c: Don't leak buf.
2021-03-18libctf: eliminate dtd_u, part 5: structs / unionsNick Alcock3-1/+3
Eliminate the dynamic member storage for structs and unions as we have for other dynamic types. This is much like the previous enum elimination, except that structs and unions are the only types for which a full-sized ctf_type_t might be needed. Up to now, this decision has been made in the individual ctf_add_{struct,union}_sized functions and duplicated in ctf_add_member_offset. The vlen machinery lets us simplify this, always allocating a ctf_lmember_t and setting the dtd_data's ctt_size to CTF_LSIZE_SENT: we figure out whether this is really justified and (almost always) repack things down into a ctf_stype_t at ctf_serialize time. This allows us to eliminate the dynamic member paths from the iterators and query functions in ctf-types.c in favour of always using the large-structure vlen stuff for dynamic types (the diff is ugly but that's just because of the volume of reindentation this calls for). This also means the large-structure vlen stuff gets more heavily tested, which is nice because it was an almost totally unused code path before now (it only kicked in for structures of size >4GiB, and how often do you see those?) The only extra complexity here is ctf_add_type. Back in the days of the nondeduplicating linker this was called a ridiculous number of times for countless identical copies of structures: eschewing the repeated lookups of the dtd in ctf_add_member_offset and adding the members directly saved an amazing amount of time. Now the nondeduplicating linker is gone, this is extreme overoptimization: we can rip out the direct addition and use ctf_member_next and ctf_add_member_offset, just like ctf_dedup_emit does. We augment a ctf_add_type test to try adding a self-referential struct, the only thing the ctf_add_type part of this change really perturbs. This completes the elimination of dtd_u. libctf/ChangeLog 2021-03-18 Nick Alcock <nick.alcock@oracle.com> * ctf-impl.h (ctf_dtdef_t) <dtu_members>: Remove. <dtd_u>: Likewise. (ctf_dmdef_t): Remove. (struct ctf_next) <u.ctn_dmd>: Remove. * ctf-create.c (INITIAL_VLEN): New, more-or-less arbitrary initial vlen size. (ctf_add_enum): Use it. (ctf_dtd_delete): Do not free the (removed) dmd; remove string refs from the vlen on struct deletion. (ctf_add_struct_sized): Populate the vlen: do it by hand if promoting forwards. Always populate the full-size lsizehi/lsizelo members. (ctf_add_union_sized): Likewise. (ctf_add_member_offset): Set up the vlen rather than the dmd. Expand it as needed, repointing string refs via ctf_str_move_pending. Add the member names as pending strings. Always populate the full-size lsizehi/lsizelo members. (membadd): Remove, folding back into... (ctf_add_type_internal): ... here, adding via an ordinary ctf_add_struct_sized and _next iteration rather than doing everything by hand. * ctf-serialize.c (ctf_copy_smembers): Remove this... (ctf_copy_lmembers): ... and this... (ctf_emit_type_sect): ... folding into here. Figure out if a ctf_stype_t is needed here, not in ctf_add_*_sized. (ctf_type_sect_size): Figure out the ctf_stype_t stuff the same way here. * ctf-types.c (ctf_member_next): Remove the dmd path and always use the vlen. Force large-structure usage for dynamic types. (ctf_type_align): Likewise. (ctf_member_info): Likewise. (ctf_type_rvisit): Likewise. * testsuite/libctf-regression/type-add-unnamed-struct-ctf.c: Add a self-referential type to this test. * testsuite/libctf-regression/type-add-unnamed-struct.c: Adjusted accordingly. * testsuite/libctf-regression/type-add-unnamed-struct.lk: Likewise.
2021-03-18libctf: eliminate dtd_u, part 4: enumsNick Alcock2-0/+111
This is the first tricky one, the first complex multi-entry vlen containing strings. To handle this in vlen form, we have to handle pending refs moving around on realloc. We grow vlen regions using a new ctf_grow_vlen function, and iterate through the existing enums every time a grow happens, telling the string machinery the distance between the old and new vlen region and letting it adjust the pending refs accordingly. (This avoids traversing all outstanding refs to find the refs that need adjusting, at the cost of having to traverse one enum: an obvious major performance win.) Addition of enums themselves (and also structs/unions later) is a bit trickier than earlier forms, because the type might be being promoted from a forward, and forwards have no vlen: so we have to spot that and create it if needed. Serialization of enums simplifies down to just telling the string machinery about the string refs; all the enum type-lookup code loses all its dynamic member lookup complexity entirely. A new test is added that iterates over (and gets values of) an enum with enough members to force a round of vlen growth. libctf/ChangeLog 2021-03-18 Nick Alcock <nick.alcock@oracle.com> * ctf-impl.h (ctf_dtdef_t) <dtd_vlen_alloc>: New. (ctf_str_move_pending): Declare. * ctf-string.c (ctf_str_add_ref_internal): Fix error return. (ctf_str_move_pending): New. * ctf-create.c (ctf_grow_vlen): New. (ctf_dtd_delete): Zero out the vlen_alloc after free. Free the vlen later: iterate over it and free enum name refs first. (ctf_add_generic): Populate dtd_vlen_alloc from vlen. (ctf_add_enum): populate the vlen; do it by hand if promoting forwards. (ctf_add_enumerator): Set up the vlen rather than the dmd. Expand it as needed, repointing string refs via ctf_str_move_pending. Add the enumerand names as pending strings. * ctf-serialize.c (ctf_copy_emembers): Remove. (ctf_emit_type_sect): Copy the vlen into place and ref the strings. * ctf-types.c (ctf_enum_next): The dynamic portion now uses the same code as the non-dynamic. (ctf_enum_name): Likewise. (ctf_enum_value): Likewise. * testsuite/libctf-lookup/enum-many-ctf.c: New test. * testsuite/libctf-lookup/enum-many.lk: New test.
2021-03-18libctf: do not corrupt strings across ctf_serializeNick Alcock2-0/+96
The preceding change revealed a new bug: the string table is sorted for better compression, so repeated serialization with type (or member) additions in the middle can move strings around. But every serialization flushes the set of refs (the memory locations that are automatically updated with a final string offset when the strtab is updated), so if we are not to have string offsets go stale, we must do all ref additions within the serialization code (which walks the complete set of types and symbols anyway). Unfortunately, we were adding one ref in another place: the type name in the dynamic type definitions, which has a ref added to it by ctf_add_generic. So adding a type, serializing (via, say, one of the ctf_write functions), adding another type with a name that sorts earlier, and serializing again will corrupt the name of the first type because it no longer had a ref pointing to its dtd entry's name when its string offset was shifted later in the strtab to mae way for the other type. To ensure that we don't miss strings, we also maintain a set of *pending refs* that will be added later (during serialization), and remove entries from that set when the ref is finally added. We always use ctf_str_add_pending outside ctf-serialize.c, ensure that ctf_serialize adds all strtab offsets as refs (even those in the dtds) on every serialization, and mandate that no refs are live on entry to ctf_serialize and that all pending refs are gone before strtab finalization. (Of necessity ctf_serialize has to traverse all strtab offsets in the dtds in order to serialize them, so adding them as refs at the same time is easy.) (Note that we still can't erase unused atoms when we roll back, though we can erase unused refs: members and enums are still not removed by rollbacks and might reference strings added after the snapshot.) libctf/ChangeLog 2021-03-18 Nick Alcock <nick.alcock@oracle.com> * ctf-hash.c (ctf_dynset_elements): New. * ctf-impl.h (ctf_dynset_elements): Declare it. (ctf_str_add_pending): Likewise. (ctf_dict_t) <ctf_str_pending_ref>: New, set of refs that must be added during serialization. * ctf-string.c (ctf_str_create_atoms): Initialize it. (CTF_STR_ADD_REF): New flag. (CTF_STR_MAKE_PROVISIONAL): Likewise. (CTF_STR_PENDING_REF): Likewise. (ctf_str_add_ref_internal): Take a flags word rather than int params. Populate, and clear out, ctf_str_pending_ref. (ctf_str_add): Adjust accordingly. (ctf_str_add_external): Likewise. (ctf_str_add_pending): New. (ctf_str_remove_ref): Also remove the potential ref if it is a pending ref. * ctf-serialize.c (ctf_serialize): Prohibit addition of strings with ctf_str_add_ref before serialization. Ensure that the ctf_str_pending_ref set is empty before strtab finalization. (ctf_emit_type_sect): Add a ref to the ctt_name. * ctf-create.c (ctf_add_generic): Add the ctt_name as a pending ref. * testsuite/libctf-writable/reserialize-strtab-corruption.*: New test.
2021-03-18libctf: fix some tabdamage and move some code aroundNick Alcock2-4/+4
ctf-link.c is unnecessarily confusing because ctf_link_lazy_open is positioned near functions that have nothing to do with opening files. Move it around, and fix some tabdamage that's crept in lately. libctf/ChangeLog 2021-03-18 Nick Alcock <nick.alcock@oracle.com> * ctf-link.c (ctf_link_lazy_open): Move up in the file, to near ctf_link_add_ctf. * ctf-lookup.c (ctf_lookup_symbol_idx): Repair tabdamage. (ctf_lookup_by_sym_or_name): Likewise. * testsuite/libctf-lookup/struct-iteration.c: Likewise. * testsuite/libctf-regression/type-add-unnamed-struct.c: Likewise.
2021-02-20libctf, include: find types of symbols by nameNick Alcock7-101/+169
The existing ctf_lookup_by_symbol and ctf_arc_lookup_symbol functions suffice to look up the types of symbols if the caller already has a symbol number. But the caller often doesn't have one of those and only knows the name of the symbol: also, in object files, the caller might not have a useful symbol number in any sense (and neither does libctf: the 'symbol number' we use in that case literally starts at 0 for the lexicographically first-sorted symbol in the symtypetab and counts those symbols, so it corresponds to nothing useful). This means that even though object files have a symtypetab (generated by the compiler or by ld -r), the only way we can look up anything in it is to iterate over all symbols in turn with ctf_symbol_next until we find the one we want. This is unhelpful and pointlessly inefficient. So add a pair of functions to look up symbols by name in a dict and in a whole archive: ctf_lookup_by_symbol_name and ctf_arc_lookup_symbol_name. These are identical to the existing functions except that they take symbol names rather than symbol numbers. To avoid insane repetition, we do some refactoring in the process, so that both ctf_lookup_by_symbol and ctf_arc_lookup_symbol turn into thin wrappers around internal functions that do both lookup by symbol index and lookup by name. This massively reduces code duplication because even the existing lookup-by-index stuff wants to use a name sometimes (when looking up in indexed sections), and the new lookup-by-name stuff has to turn it into an index sometimes (when looking up in non-indexed sections): doing it this way lets us share most of that. The actual name->index lookup is done by ctf_lookup_symbol_idx. We do not anticipate this lookup to be as heavily used as ld.so symbol lookup by many orders of magnitude, so using the ELF symbol hashes would probably take more time to read them than is saved by using the hashes, and it adds a lot of complexity. Instead, do a linear search for the symbol name, caching all the name -> index mappings as we go, so that future searches are likely to hit in the cache. To avoid having to repeat this search over and over in a CTF archive when ctf_arc_lookup_symbol_name is used, have cached archive lookups (the sort done by ctf_arc_lookup_symbol* and the ctf_archive_next iterator) pick out the first dict they cache in a given archive and store it in a new ctf_archive field, ctfi_crossdict_cache. This can be used to store cross-dictionary cached state that depends on things like the ELF symbol table rather than the contents of any one dict. ctf_lookup_symbol_idx then caches its name->index mappings in the dictionary named in the crossdict cache, if any, so that ctf_lookup_symbol_idx in other dicts in the same archive benefit from the previous linear search, and the symtab only needs to be scanned at most once. (Note that if you call ctf_lookup_by_symbol_name in one specific dict, and then follow it with a ctf_arc_lookup_symbol_name, the former will not use the crossdict cache because it's only populated by the dict opens in ctf_arc_lookup_symbol_name. This is harmless except for a small one-off waste of memory and time: it's only a cache, after all. We can fix this later by using the archive caching machinery more aggressively.) In ctf-archive, we do similar things, turning ctf_arc_lookup_symbol into a wrapper around a new function that does both index -> ID and name -> ID lookups across all dicts in an archive. We add a new ctfi_symnamedicts cache that maps symbol names to the ctf_dict_t * that it was found in (so that linear searches for symbols don't need to be repeated): but we also *remove* a cache, the ctfi_syms cache that was memoizing the actual ctf_id_t returned from every call to ctf_arc_lookup_symbol. This is pointless: all it saves is one call to ctf_lookup_by_symbol, and that's basically an array lookup and nothing more so isn't worth caching. (Equally, given that symbol -> index mappings are cached by ctf_lookup_by_symbol_name, those calls are nearly free after the first call, so there's no point caching the ctf_id_t in that case either.) We fix up one test that was doing manual symbol lookup to use ctf_arc_lookup_symbol instead, and enhance it to check that the caching layer is not totally broken: we also add a new test to do lookups in a .o file, and another to do lookups in an archive with conflicted types and make sure that sort of multi-dict lookup is actually working. include/ChangeLog 2021-02-17 Nick Alcock <nick.alcock@oracle.com> * ctf-api.h (ctf_arc_lookup_symbol_name): New. (ctf_lookup_by_symbol_name): Likewise. libctf/ChangeLog 2021-02-17 Nick Alcock <nick.alcock@oracle.com> * ctf-impl.h (ctf_dict_t) <ctf_symhash>: New. <ctf_symhash_latest>: Likewise. (struct ctf_archive_internal) <ctfi_crossdict_cache>: New. <ctfi_symnamedicts>: New. <ctfi_syms>: Remove. (ctf_lookup_symbol_name): Remove. * ctf-lookup.c (ctf_lookup_symbol_name): Propagate errors from parent properly. Make static. (ctf_lookup_symbol_idx): New, linear search for the symbol name, cached in the crossdict cache's ctf_symhash (if available), or this dict's (otherwise). (ctf_try_lookup_indexed): Allow the symname to be passed in. (ctf_lookup_by_symbol): Turn into a wrapper around... (ctf_lookup_by_sym_or_name): ... this, supporting name lookup too, using ctf_lookup_symbol_idx in non-writable dicts. Special-case name lookup in dynamic dicts without reported symbols, which have no symtab or dynsymidx but where name lookup should still work. (ctf_lookup_by_symbol_name): New, another wrapper. * ctf-archive.c (enosym): Note that this is present in ctfi_symnamedicts too. (ctf_arc_close): Adjust for removal of ctfi_syms. Free the ctfi_symnamedicts. (ctf_arc_flush_caches): Likewise. (ctf_dict_open_cached): Memoize the first cached dict in the crossdict cache. (ctf_arc_lookup_symbol): Turn into a wrapper around... (ctf_arc_lookup_sym_or_name): ... this. No longer cache ctf_id_t lookups: just call ctf_lookup_by_symbol as needed (but still cache the dicts those lookups succeed in). Add lookup-by-name support, with dicts of successful lookups cached in ctfi_symnamedicts. Refactor the caching code a bit. (ctf_arc_lookup_symbol_name): New, another wrapper. * ctf-open.c (ctf_dict_close): Free the ctf_symhash. * libctf.ver (LIBCTF_1.2): New version. Add ctf_lookup_by_symbol_name, ctf_arc_lookup_symbol_name. * testsuite/libctf-lookup/enum-symbol.c (main): Use ctf_arc_lookup_symbol rather than looking up the name ourselves. Fish it out repeatedly, to make sure that symbol caching isn't broken. (symidx_64): Remove. (symidx_32): Remove. * testsuite/libctf-lookup/enum-symbol-obj.lk: Test symbol lookup in an unlinked object file (indexed symtypetab sections only). * testsuite/libctf-writable/symtypetab-nonlinker-writeout.c (try_maybe_reporting): Check symbol types via ctf_lookup_by_symbol_name as well as ctf_symbol_next. * testsuite/libctf-lookup/conflicting-type-syms.*: New test of lookups in a multi-dict archive.
2021-02-20Include ld-lib.exp from ctf-lib.expAlan Modra2-172/+3
* testsuite/config/default.exp (ld_L_opt): Define. * testsuite/lib/ctf-lib.exp (load_common_lib): Delete. Instead load ld-lib.exp. (run_host_cmd, run_host_cmd_yesno, check_compiler_available): Delete. (compile_one_cc, check_ctf_available): Delete.
2021-02-04libctf, ld: fix symtypetab and var section population under ld -rNick Alcock9-1/+420
The variable section in a CTF dict is meant to contain the types of variables that do not appear in the symbol table (mostly file-scope static declarations). We implement this by having the compiler emit all potential data symbols into both sections, then delete those symbols from the variable section that correspond to data symbols the linker has reported. Unfortunately, the check for this in ctf_serialize is wrong: rather than checking the set of linker-reported symbols, we check the set of names in the data object symtypetab section: if the linker has reported no symbols at all (usually if ld -r has been run, or if a non-linker program that does not use symbol tables is calling ctf_link) this will include every single symbol, emptying the variable section completely. Worse, when ld -r is in use, we want to force writeout of every symtypetab entry on the inputs, in an indexed section, whether or not the linker has reported them, since this isn't a final link yet and the symbol table is not finalized (and may grow more symbols than the linker has yet reported). But the check for this is flawed too: we were relying on ctf_link_shuffle_syms not having been called if no symbols exist, but that function is *always* called by ld even when ld -r is in use: ctf_link_add_linker_symbol is the one that's not called when there are no symbols. We clearly need to rethink this. Using the emptiness of the set of reported symbols as a test for ld -r is just ugly: the linker already knows if ld -r is underway and can just tell us. So add a new linker flag CTF_LINK_NO_FILTER_REPORTED_SYMS that is set to stop the linker filtering the symbols in the symtypetab sections using the set that the linker has reported: use the presence or absence of this flag to determine whether to emit unindexed symtabs: we only remove entries from the variable section when filtering symbols, and we only remove them if they are in the reported symbol set, fixing the case where no symbols are reported by the linker at all. (The negative sense of the new CTF_LINK flag is intentional: the common case, both for ld and for simple tools that want to do a ctf_link with no ELF symbol table in sight, is probably to filter out symbols that no linker has reported: i.e., for the simple tools, all of them.) There's another wrinkle, though. It is quite possible for a non-linker to add symbols to a dict via ctf_add_*_sym and then write it out via the ctf_write APIs: perhaps it's preparing a dict for a later linker invocation. Right now this would not lead to anything terribly meaningful happening: ctf_serialize just assumes it was called via ctf_link if symbols are present. So add an (internal-to-libctf) flag that indicates that a writeout is happening via ctf_link_write, and set it there (propagating it to child dicts as needed). ctf_serialize can then spot when it is not being called by a linker, and arrange to always write out an indexed, sorted symtypetab for fastest possible future symbol lookup by name in that case. (The writeouts done by ld -r are unsorted, because the only thing likely to use those symtabs is the linker, which doesn't benefit from symtypetab sorting.) Tests added for all three linking cases (ld -r, ld -shared, ld), with a bit of testsuite framework enhancement to stop it unconditionally linking the CTF to be checked by the lookup program with -shared, so tests can now examine CTF linked with -r or indeed with no flags at all, though the output filename is still foo.so even in this case. Another test added for the non-linker case that endeavours to determine whether the symtypetab is sorted by examining the order of entries returned from ctf_symbol_next: nobody outside libctf should rely on this ordering, but this test is not outside libctf :) include/ChangeLog 2021-01-26 Nick Alcock <nick.alcock@oracle.com> * ctf-api.h (CTF_LINK_NO_FILTER_REPORTED_SYMS): New. ld/ChangeLog 2021-01-26 Nick Alcock <nick.alcock@oracle.com> * ldlang.c (lang_merge_ctf): Set CTF_LINK_NO_FILTER_REPORTED_SYMS when appropriate. libctf/ChangeLog 2021-01-27 Nick Alcock <nick.alcock@oracle.com> * ctf-impl.c (_libctf_nonnull_): Add parameters. (LCTF_LINKING): New flag. (ctf_dict_t) <ctf_link_flags>: Mention it. * ctf-link.c (ctf_link): Keep LCTF_LINKING set across call. (ctf_write): Likewise, including in child dictionaries. (ctf_link_shuffle_syms): Make sure ctf_dynsyms is NULL if there are no reported symbols. * ctf-create.c (symtypetab_delete_nonstatic_vars): Make sure the variable has been reported as a symbol by the linker. (symtypetab_skippable): Mention relationship between SYMFP and the flags. (symtypetab_density): Adjust nonnullity. Exit early if no symbols were reported and force-indexing is off (i.e., we are doing a final link). (ctf_serialize): Handle the !LCTF_LINKING case by writing out an indexed, sorted symtypetab (and allow SYMFP to be NULL in this case). Turn sorting off if this is a non-final link. Only delete nonstatic vars if we are filtering symbols and the linker has reported some. * testsuite/libctf-regression/nonstatic-var-section-ld-r*: New test of variable and symtypetab section population when ld -r is used. * testsuite/libctf-regression/nonstatic-var-section-ld-executable.lk: Likewise, when ld of an executable is used. * testsuite/libctf-regression/nonstatic-var-section-ld.lk: Likewise, when ld -shared alone is used. * testsuite/libctf-regression/nonstatic-var-section-ld*.c: Lookup programs for the above. * testsuite/libctf-writable/symtypetab-nonlinker-writeout.*: New test, testing survival of symbols across ctf_write paths. * testsuite/lib/ctf-lib.exp (run_lookup_test): New option, nonshared, suppressing linking of the SOURCE with -shared.
2021-01-19libctf, create: fix ctf_type_add of structs with unnamed membersNick Alcock3-0/+94
Our recent commit to support unnamed structure members better ditched the old ctf_member_iter iterator body in favour of ctf_member_next. However, these functions treat unnamed structure members differently: ctf_member_iter just returned whatever the internal representation contained, while ctf_member_next took care to always return "" rather than sometimes returning "" and sometimes NULL depending on whether the dict was dynamic (a product of ctf_create) or not (a product of ctf_open). After this commit, ctf_member_iter did the same. It was always a bug for external callers not to treat a "" return from these functions as if it were NULL, so only buggy callers could be affected -- but one of those buggy callers was ctf_add_type, which assumed that it could just take whatever name was returned from ctf_member_iter and slam it directly into the internal representation of a dynamic dict -- which expects NULL for unnamed members, not "". The net effect of all of this is that taking a struct containing unnamed members and ctf_add_type'ing it into a dynamic dict produced a dict whose unnamed members were inaccessible to ctf_member_info (though if you wrote that dict out and then ctf_open'ed it, they would magically reappear again). Compensate for this by suitably transforming a "" name into NULL in the internal representation, as should have been done all along. libctf/ChangeLog 2021-01-19 Nick Alcock <nick.alcock@oracle.com> * ctf-create.c (membadd): Transform ""-named members into NULL-named ones. * testsuite/libctf-regression/type-add-unnamed-struct*: New test.
2021-01-19libctf: lookup_by_name: do not return success for nonexistent pointer typesNick Alcock3-2/+13
The recent work allowing lookups of pointers in child dicts when the pointed-to type is in the parent dict broke the case where a pointer type that does not exist at all is looked up: we mistakenly return the pointed-to type, which is likely not a pointer at all. This causes considerable confusion. Fixed, with a new testcase. libctf/ChangeLog 2021-01-19 Nick Alcock <nick.alcock@oracle.com> * ctf-lookup.c (ctf_lookup_by_name_internal): Do not return the base type if looking up a nonexistent pointer type. * testsuite/libctf-regression/pptrtab*: Test it.
2021-01-05libctf, testsuite: adjust for real return type of ctf_member_countNick Alcock1-3/+3
This returns an int, not a long int or an ssize_t (as one test was inconsistently assuming). libctf/ChangeLog 2021-01-05 Nick Alcock <nick.alcock@oracle.com> * testsuite/libctf-lookup/struct-iteration.c (main): ctf_member_count returns an int.
2021-01-05libctf, testsuite: don't run without a suitable compilerNick Alcock3-1/+16
We never actually check to see if the compiler supports CTF, or even if a suitable compiler exists. libctf/ChangeLog 2021-01-05 Nick Alcock <nick.alcock@oracle.com> * Makefile.am (BASEDIR): New. (BFDDIR): Likewise. (check-DEJAGNU): Add development.exp to prerequisites. (development.exp): New. (CONFIG_STATUS_DEPENDENCIES): New. (EXTRA_DEJAGNU_SITE_CONFIG): Likewise. (DISTCLEANFILES): Likewise. * Makefile.in: Regenerated. * testsuite/lib/ctf-lib.exp (check_ctf_available): Return boolean. * testsuite/libctf-lookup/lookup.exp: Call check_ctf_available. * testsuite/libctf-regression/regression.exp: Likewise.
2021-01-05libctf: fix lookups of pointers by name in parent dictsNick Alcock8-0/+258
When you look up a type by name using ctf_lookup_by_name, in most cases libctf can just strip off any qualifiers and look for the name, but for pointer types this doesn't work, since the caller will want the pointer type itself. But pointer types are nameless, and while they cite the types they point to, looking up a type by name requires a link going the *other way*, from the type pointed to to the pointer type that points to it. libctf has always built this up at open time: ctf_ptrtab is an array of type indexes pointing from the index of every type to the index of the type that points to it. But because it is built up at open time (and because it uses type indexes and not type IDs) it is restricted to working within a single dict and ignoring parent/child relationships. This is normally invisible, unless you manage to get a dict with a type in the parent but the only pointer to it in a child. The ctf_ptrtab will not track this relationship, so lookups of this pointer type by name will fail. Since which type is in the parent and which in the child is largely opaque to the user (which goes where is up to the deduplicator, and it can and does reshuffle things to save space), this leads to a very bad user experience, with an obviously-visible pointer type which ctf_lookup_by_name claims doesn't exist. The fix is to have another array, ctf_pptrtab, which is populated in child dicts: like the parent's ctf_ptrtab, it has one element per type in the parent, but is all zeroes except for those types which are pointed to by types in the child: so it maps parent dict indices to child dict indices. The array is grown, and new child types scanned, whenever a lookup happens and new types have been added to the child since the last time a lookup happened that might need the pptrtab. (So for non-writable dicts, this only happens once, since new types cannot be added to non-writable dicts at all.) Since this introduces new complexity (involving updating only part of the ctf_pptrtab) which is only seen when a writable dict is in use, we introduce a new libctf-writable testsuite that contains lookup tests with no corresponding CTF-containing .c files (which can thus be run even on platforms with no .ctf-section support in the linker yet), and add a test to check that creation of pointers in children to types in parents and a following lookup by name works as expected. The non- writable case is tested in a new libctf-regression testsuite which is used to track now-fixed outright bugs in libctf. libctf/ChangeLog 2021-01-05 Nick Alcock <nick.alcock@oracle.com> * ctf-impl.h (ctf_dict_t) <ctf_pptrtab>: New. <ctf_pptrtab_len>: New. <ctf_pptrtab_typemax>: New. * ctf-create.c (ctf_serialize): Update accordingly. (ctf_add_reftype): Note that we don't need to update pptrtab here, despite updating ptrtab. * ctf-open.c (ctf_dict_close): Destroy the pptrtab. (ctf_import): Likewise. (ctf_import_unref): Likewise. * ctf-lookup.c (grow_pptrtab): New. (refresh_pptrtab): New, update a pptrtab. (ctf_lookup_by_name): Turn into a wrapper around (and rename to)... (ctf_lookup_by_name_internal): ... this: construct the pptrtab, and use it in addition to the parent's ptrtab when parent dicts are searched. * testsuite/libctf-regression/regression.exp: New testsuite for regression tests. * testsuite/libctf-regression/pptrtab*: New test. * testsuite/libctf-writable/writable.exp: New testsuite for tests of writable CTF dicts. * testsuite/libctf-writable/pptrtab*: New test.
2021-01-05libctf, include: support unnamed structure members betterNick Alcock5-0/+208
libctf has no intrinsic support for the GCC unnamed structure member extension. This principally means that you can't look up named members inside unnamed struct or union members via ctf_member_info: you have to tiresomely find out the type ID of the unnamed members via iteration, then look in each of these. This is ridiculous. Fix it by extending ctf_member_info so that it recurses into unnamed members for you: this is still unambiguous because GCC won't let you create ambiguously-named members even in the presence of this extension. For consistency, and because the release hasn't happened and we can still do this, break the ctf_member_next API and add flags: we specify one flag, CTF_MN_RECURSE, which if set causes ctf_member_next to automatically recurse into unnamed members for you, returning not only the members themselves but all their contained members, so that you can use ctf_member_next to identify every member that it would be valid to call ctf_member_info with. New lookup tests are added for all of this. include/ChangeLog 2021-01-05 Nick Alcock <nick.alcock@oracle.com> * ctf-api.h (CTF_MN_RECURSE): New. (ctf_member_next): Add flags argument. libctf/ChangeLog 2021-01-05 Nick Alcock <nick.alcock@oracle.com> * ctf-impl.h (struct ctf_next) <u.ctn_next>: Move to... <ctn_next>: ... here. * ctf-util.c (ctf_next_destroy): Unconditionally destroy it. * ctf-lookup.c (ctf_symbol_next): Adjust accordingly. * ctf-types.c (ctf_member_iter): Reimplement in terms of... (ctf_member_next): ... this. Support recursive unnamed member iteration (off by default). (ctf_member_info): Look up members in unnamed sub-structs. * ctf-dedup.c (ctf_dedup_rhash_type): Adjust ctf_member_next call. (ctf_dedup_emit_struct_members): Likewise. * testsuite/libctf-lookup/struct-iteration-ctf.c: Test empty unnamed members, and a normal member after the end. * testsuite/libctf-lookup/struct-iteration.c: Verify that ctf_member_count is consistent with the number of successful returns from a non-recursive ctf_member_next. * testsuite/libctf-lookup/struct-iteration-*: New, test iteration over struct members. * testsuite/libctf-lookup/struct-lookup.c: New test. * testsuite/libctf-lookup/struct-lookup.lk: New test.