aboutsummaryrefslogtreecommitdiff
path: root/libctf
AgeCommit message (Collapse)AuthorFilesLines
3 dayslibctf: doc: add __float128 and SIMD vector classification to spec.Bruce McCulloch1-2/+9
This patch adds two additional distinct types (__float128 and the SIMD vector type generated from the vector_size attribute) to the umbrella of two existing types (long double and array, respectively). These types were previously invalid, producing CTF_K_UNKNOWN in the case of __float128 or a float in the case of the SIMD vector. This patch will cleanly allow these types to be represented more accurately without breaking back-compat. Signed-off-by: Bruce McCulloch <bruce.mcculloch@oracle.com> Reviewed-by: Nick Alcock <nick.alcock@oracle.com>
3 dayslibctf: add root-visibility-addition testNick Alcock2-0/+39
libctf/ * testsuite/libctf-writable/ctf-nonroot-addition.*: New test.
3 dayslibctf: create: check the right root-visible flag when adding enumerandsNick Alcock1-1/+1
The root-visible flag we're dealing with here is directly out of the dict, not a flag passed in to the API, so it does not have the values CTF_ADD_ROOT or CTF_ADD_NONROOT: instead it's simply zero for non-root-visible, nonzero otherwise. Fix the test. libctf/ * ctf-create.c (ctf_add_enumerator): Fix root-visibility test.
3 dayslibctf: create: addition of non-root types should not return root typesNick Alcock1-3/+3
If you add a non-root type to a dict, you should always get a new, unique type ID back, even if a root-visible type with the same name already exists. Unfortunately, if the root-visible type is a forward, and you're adding a non-root-visible struct, union, or enum, the machinery to detect forwards and promote them to the concrete type fires in this case and returns the root-visible type! If this is an enum being inserted hidden because its enumerands conflict with some other enum, this will lead to failure later on: in any case, it's seriously counterintuitive to add a non-root- visible type and get a root-visible one instead. Fix this by checking the root-visible flag properly and only checking for forwards if this type is root-visible. (This may lead to a certain degree of proliferation of non-root-visible forwards: we can add a cleanup pass for those later if needed.) libctf/ * ctf-create.c (ctf_add_struct_sized): Check the root-visible flag when doing forward promotion. (ctf_add_union_sized): Likewise. (ctf_add_enum): Likewise. Reviewed-by: Bruce McCulloch <bruce.mcculloch@oracle.com>
2025-06-04libctf: use __attribute__((__gnu_printf__)) where appropriateNick Alcock1-0/+5
We don't use any GNU-specific printf args, but this prevents warnings about %z, observed on MinGW even though every libc anyone is likely to use there supports %z perfectly well, and we're not stopping using it just because MinGW complains. Doing this means we stand more chance of seeing *actual* problems on such platforms without them being drowned in noise. We turn this off on clang, which doesn't support __gnu_printf__. Suggested by Eli Zaretskii. libctf/ PR libctf/31863 * ctf-impl.h (_libctf_printflike_): Use __gnu_printf__.
2025-06-04libctf, dedup: reclaim space wasted by duplicate hidden typesNick Alcock1-12/+28
In normal deduplicating links, we insert every type (identified by its unique hash) precisely once. But conflicting types appear in multiple dicts, so for those, we loop, inserting them into every target dict in turn (each corresponding to an input dict that type appears in). But in cu-mapped links, some of those dicts may have been merged into one: now that we are hiding duplicate conflicting types more aggressively in such links, we are getting duplicate identical hidden types turning up in large numbers. Fix this by eliminating them in cu-mapping phase 1 (the phase in which this merging takes place), by checking to see if a type with this hash has already been inserted in this dict and skipping it if so. This is redundant and a waste of time in other cu-mapping phases and in normal links, but in cu-mapped links it saves a few tens to hundreds of kilobytes in kernel-sized links. libctf/ PR libctf/33047 * ctf-dedup.c (ctf_dedup_emit_type): Check for already-emitted types in cu-mapping phase 1.
2025-06-04libctf: dedup: preserve non-root flag across normal linksNick Alcock6-56/+322
The previous commits dropped preservation of the non-root flag in ctf_link and arranged to use it somewhat differently to track conflicting types in cu-mapped CUs when doing cu-mapped links. This was necessary to prevent entirely spuriously hidden types from appearing on the output of such links. Bring it (and the test for it) back. The problem with the previous design was that it implicitly assumed that the non-root flag it saw on the input was always meant to be preserved (when in the final phase of cu-mapped links it merely means that conflicting types were found in intermediate links), and also that it could figure out what the non-root flag on the input was by sucking in the non-root flag of the input type corresponding to an output in the output mapping (which maps type hashes to a corresponding type on some input). This method of getting properties of the input type *does* work *if* that property was one of those hashed by the ctf_dedup_hash_type process. In that case, every type with a given hash will have the same value for all hashed-in properties, so it doesn't matter which one is consulted (the output mapping points at an arbitrary one of those input types). But the non-root flag is explicitly *not* hashed in: as a comment in ctf_dedup_rhash_type notes, being non-root is not a property of a type, and two types (one non-root, one not) can perfectly well be the same type even though one is visible and one isn't. So just copying the non-root flag from the output mapping's idea of the input type will copy in a value that is not stabilized by the hash, so is more-or-less random! So we cannot do that. We have to do something else, which means we have to decide what to do if two identical types with different nonroot flag values pop up. The most sensible thing to do is probably to say that if all instances of a type are non-root-visible, the linked output should also be non-root-visible: any root-visible types in that set, and the output type is root-visible again. We implement this with a new cd_nonroot_consistency dynhash, which maps type hashes to the value 0 ("all instances root-visible"), 1 ("all instances non-root-visible") or 2 ("inconsistent"). After hashing is over, we save a bit of memory by deleting everything from this hashtab that doesn't have a value of 1 ("non-root-visible"), then use this to decide whether to emit any given type as non-root-visible or not. However... that's not quite enough. In cu-mapped links, we want to disregard this whole thing because we just hide everything -- but in phase 2, when we take the smushed-together CUs resulting from phase 1 and deduplicate them against each other, we want to do what the previous commits implemented and ignore the non-root flag entirely, instead falling back to preventing clashes by hiding anything that would be considered conflicting. We extend the existing cu_mapped parameter to various bits of ctf_dedup so that it is now tristate: 0 means a normal link, 1 means the smush-it- together phase of cu-mapped links, and 2 means the final phase of cu-mapped links. We do the hide-conflicting stuff only in phase 2, meaning that normal links by GNU ld can always respect the value of the nonroot flag put on types in the input. (One extra thing added as part of this: you can now efficiently delete the last value returned by ctf_dynhash_next() by calling ctf_dynhash_next_remove.) We bring back the ctf-nonroot-linking test with one tweak: linking now works on mingw as long as you're using the ucrt libc, so re-enable it for better test coverage on that platform. libctf/ PR libctf/33047 * ctf-hash.c (ctf_dynhash_next_remove): New. * ctf-impl.h (struct ctf_dedup) [cd_nonroot_consistency]: New. * ctf-link.c (ctf_link_deduplicating): Differentiate between cu-mapped and non-cu-mapped links, even in the final phase. * ctf-dedup.c (ctf_dedup_hash_type): Callback prototype addition. Get the non-root flag and pass it down. (ctf_dedup_rhash_type): Callback prototype addition. Document restrictions on use of the nonroot flag. (ctf_dedup_populate_mappings): Populate cd_nonroot_consistency. (ctf_dedup_hash_type_fini): New function: delete now-unnecessary values from cd_nonroot_consistency. (ctf_dedup_init): Initialize it. (ctf_dedup_fini): Destroy it. (ctf_dedup): cu_mapping is now cu_mapping_phase. Call ctf_dedup_hash_type_fini. (ctf_dedup_emit_type): Use cu_mapping_phase and cd_nonroot_consistency to propagate the non-root flag into outputs for normal links, and to do name-based conflict checking only for phase 2 of cu-mapped links. (ctf_dedup_emit): cu_mapping is now cu_mapping_phase. Adjust assertion accordingly. * testsuite/libctf-writable/ctf-nonroot-linking.c: Bring back. * testsuite/libctf-writable/ctf-nonroot-linking.lk: Likewise.
2025-06-04libctf: dedup: improve hiding of conflicting types in the same dictNick Alcock1-5/+40
If types are conflicting, they are usually moved into separate child dicts -- but not always. If they are added to the same dict by the cu-mapping mechanism (as used e.g. for multi-TU kernel modules), we can easily end up adding multiple conflicting types with the same name to the same dict. The mechanism used for turning on the non-root-visible flag in order to do this had a kludge attached which always hid types with the same name, whether or not they were conflicting. This is unnecessary and can hide types that should not be hidden, as well as hiding bugs. Remove it, and replace it with two different approaches: - for everything but cu-mapped links (the in-memory first phase of a link with ctf_link_add_cu_mapping in force), check for duplicate names if types are conflicting, and mark them as hidden if the names are found. This will never happen in normal links (in an upcoming commit we will suppress doing even this much in such cases). - for cu-mapped links, the only case that merges multiple distinct target dicts into one, we apply a big hammer and simply hide everything! The non-root flag will be ignored in the next link phase anyway (which dedups the cu-mapped pieces against each other), and this way we can be sure that merging multiple types cannot incur name clashes at this stage. The result seems to work: the only annoyance is that when enums with conflicting enumerators are found in a single cu-mapped child (so, really multiple merged children), you may end up with every instance of that enum being hidden for reasons of conflictingness. I don't see a real way to avoid that. libctf/ PR libctf/33047 * ctf-dedup.c (ctf_dedup_emit_type): Only consider non conflicting types. Improve type hiding in the presence of clashing enumerators. Hide everything when doing a cu-mapped link: they will be unhidden by the next link pass if nonconflicting.
2025-06-04Revert "libctf: fix linking of non-root-visible types"Nick Alcock3-132/+2
This reverts commit 87b2f673102884d7c69144c85a26ed5dbaa4f86a. It is based on a misconception, that hidden types in the deduplicator input should always be hidden in the output. For cu-mapped links, and final links following cu-mapped links, this is not true: we want to hide inputs if they were conflicting on the output and no more. We will reintroduce the testcase once a better fix is found. libctf/ PR libctf/33047 * ctf-dedup.c (ctf_dedup_emit_type): Don't respect the nonroot flag. * testsuite/libctf-writable/ctf-nonroot-linking.c: Removed. * testsuite/libctf-writable/ctf-nonroot-linking.lk: Removed.
2025-05-23libctf: archive, open: when opening, always set errp to somethingNick Alcock4-2/+28
ctf_arc_import_parent, called by the cached-opening machinery used by ctf_archive_next and archive-wide lookup functions like ctf_arc_lookup_symbol, has an err-pointer parameter like all other opening functions. Unfortunately it unconditionally initializes it whenever provided, even if there was no error, which can lead to its being initialized to an uninitialized value. This is not technically an API-contract violation, since we don't define what happens to the error value except when an error happens, but it is still unpleasant. Initialize it only when there is an actual error, so we never initialize it to an uninitialized value. While we're at it, improve all the opening pathways: on success, set errp to 0, rather than leaving it what it was, reducing the likelihood of uninitialized error param returns in callers too. (This is inconsistent with the treatment of ctf_errno(), but the err value being a parameter passed in from outside makes the divergence acceptable: in open functions, you're never going to be overwriting some old error value someone might want to keep around across multiple calls, some of which are successful and some of which are not.) Soup up existing tests to verify all this. Thanks to Bruce McCulloch for the original patch, and Stephen Brennan for the report. libctf/ PR libctf/32903 * ctf-archive.c (ctf_arc_open_internal): Zero errp on success. (ctf_dict_open_sections): Zero errp at the start. (ctf_arc_import_parent): Intialize err. * ctf-open.c (ctf_bufopen): Zero errp at the start. * testsuite/libctf-lookup/add-to-opened.c: Make sure one-element archive opens update errp. * testsuite/libctf-writable/ctf-compressed.c: Make sure real archive opens update errp.
2025-02-28libctf: fix cv-qualified unnamed struct/union field lookupNick Alcock4-13/+55
GCC permits not only unnamed structs and unions, but cv-qualified ones. Our earlier fix in 6c3a38777b38a2ad87e2b2bcec4567578d1c83ec supported unnamed structs and unions, but only unqualified ones. Resolving away cvr-quals of nameless fields (and, irrelevantly, typedefs) is easy and fixes this problem. Tests adjusted accordingly. libctf/ PR libctf/32746 * ctf-types.c (ctf_member_next): Resolve away cv-quals. (ctf_member_info): Likewise. * testsuite/libctf-lookup/struct-iteration-ctf.c: Add a cv-qualified type or two: make sure to keep a non-qualified one. * testsuite/libctf-lookup/struct-iteration.c: Verify consistency of ctf_member_next and ctf_member_info. * testsuite/libctf-lookup/struct-iteration.lk: Adjust. Tested-by: Stephen Brennan <stephen.s.brennan@oracle.com>
2025-02-28libctf: fix slices of slices and of enumsNick Alcock4-6/+98
Slices had a bunch of horrible usability problems. In particular, while towers of cv-quals are resolved away by functions that need to do it, towers of cv-quals with slices in the middle are not resolved away by functions like ctf_enum_value that can see through slices: resolving volatile -> slice -> const -> enum will leave it with a 'const', which will error pointlessly, annoying callers, who reasonably expect slices to be more invisible than this. (The user-callable ctf_type_resolve still does not resolve away slices, because this is the only way users can see that the slices are there at all.) This is induced by a fix for another wart: ctf_add_enumerator does not resolve anything away at all, so you can't even add enumerators to const or volatile enums -- and more problematically, you can't add enumerators to enums with an explicit encoding without resolving away the types by hand, since ctf_add_enum_encoded works by returning a slice! ctf_add_enumerator now resolves away all of those, so any cvr-or-typedef-or-slice-qual terminating in an enum can be added to, exactly as callers likely expect. (New tests added.) libctf/ * ctf-create.c (ctf_add_enumerator): Resolve away cvr-qualness. * ctf-types.c (ctf_type_resolve_unsliced): Don't terminate at the first slice. * testsuite/libctf-writable/slice-of-slice.*: New test.
2025-02-28Remove unnecessary non-standard & unportable inclusions.Jonas 'Sortie' Termansen2-2/+0
<memory.h> is not needed and not standardized and is just an alias for <string.h>. <sys/param.h> is not needed and not standardized and contains a kitchen sink of various unportable definitions not agreed upon and best done manually or through other headers. These fixes are needed to compile binutils on Sortix and other operating systems with a strict POSIX.1-2024 libc without obsolete features. Signed-off-by: Jonas 'Sortie' Termansen <sortie@maxsi.org>
2025-02-25[libctf] Fix warning: @xref should not appear on @multitable lineTom de Vries1-1/+1
When building gdb, I run into: ... ctf-spec.texi:809: warning: @xref should not appear on @multitable line ... The line in question is: ... @multitable {Kind} {@code{CTF_K_VOLATILE}} {Indicates a type that cannot be represented in CTF, or that} {@xref{Pointers typedefs and cvr-quals}} ... which defines a prototype row with 4 columns. However, the table only has 3 colums: ... @headitem Kind @tab Macro @tab Purpose ... Fix the warning by removing the item in the prototype row representing a fourth column. Tested on aarch64-linux. PR libctf/32044 Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=32044
2025-01-19Add markers for bihnutils 2.44 branchNick Clifton1-0/+4
2025-01-01Update year range in copyright notice of binutils filesAlan Modra40-41/+41
2024-09-24binutils testsuite: canonicalize subtest names in libctfFrank Ch. Eigler2-9/+8
Previous code included the full $srcdir pathnames in the individual subtest PASS/FAIL names, which makes it difficult to compute comparisons or regressions between test runs on different machines. This version switches to the basename only, which are common. Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
2024-08-29PR32093, -Walloc-size warning in ctf-hash.cAlan Modra1-2/+8
PR 32093 * ctf-hash.c (ctf_dynhash_create_sized, ctf_hashtab_insert): Avoid -Walloc-size warning.
2024-07-31libctf: fix ctf_archive_count return value on big-endianNick Alcock1-1/+1
This failed to properly byteswap its return value. The ctf_archive format predates the idea of "just write natively and flip on open", and byteswaps all over the place. It's too easy to forget one. The next revision of the archive format (not versioned, so we just tweak the magic number instead) should be native-endianned like the dicts inside it are. libctf/ * ctf-archive.c (ctf_archive_count): Byteswap return value.
2024-07-31libctf: dump: fix small leakNick Alcock1-1/+2
If you asprintf something and then use it only as input to another asprintf, it helps to free it afterwards. libctf/ * ctf-dump.c (ctf_dump_header): Free the flagstr after use. (ctf_dump): Make a NULL return slightly clearer.
2024-07-31libctf: fix ref leak of names of newly-inserted non-root-visible typesNick Alcock1-3/+3
A bug in ctf_dtd_delete led to refs in the string table to the names of non-root-visible types not being removed when the DTD was. This seems harmless, but actually it would lead to a write down a pointer into freed memory if such a type was ctf_rollback()ed over and then the dict was serialized (updating all the refs as the strtab was serialized in turn). Bug introduced in commit fe4c2d55634c700ba527ac4183e05c66e9f93c62 ("libctf: create: non-root-visible types should not appear in name tables") which is included in binutils 2.35. libctf/ * ctf-create.c (ctf_dtd_delete): Remove refs for all types with names, not just root-visible ones.
2024-07-31libctf: clean up hashtab error handling messNick Alcock2-11/+16
The dict and archive opening code in libctf is somewhat unusual, because unlike everything else, it cannot report errors by setting an error on the dict, because in case of error there isn't one. They get passed an error integer pointer that is set on error instead. Inside ctf_bufopen this is implemented by calling ctf_set_open_errno and passing it a positive error value. In turn this means that most things it calls (including init_static_types) return zero on success and a *positive* ECTF_* or errno value on error. This trickles down to ctf_dynhash_insert_type, which is used by init_static_types to add newly-detected types to the name tables. This was returning the error value it received from a variety of functions without alteration. ctf_dynhash_insert conformed to this contract by returning a positive value on error (usually OOM), which is unfortunate for multiple reasons: - ctf_dynset_insert returns a *negative* value - ctf_dynhash_insert and ctf_dynset_insert don't take an fp, so the value they return is turned into the errno, so it had better be right, callers don't just check for != 0 here - more or less every single caller of ctf_dyn*_insert in libctf other than ctf_dynhash_insert_type (and there are a *lot*, mostly in the deduplicator) assumes that ctf_dynhash_insert returns a negative value on error, even though it doesn't. In practice the only possible error is OOM, but if OOM does happen we end up with a nonsense error value. The simplest fix for this seems to be to make ctf_dynhash_insert and ctf_dynset_insert conform to the usual interface contract: negative values are errors. This in turn means that ctf_dynhash_insert_type needs to change: let's make it consistent too, returning a negative value on error, putting the error on the fp in non-negated form. init_static_types_internal adapts to this by negating the error return from ctf_dynhash_insert_type, so the value handed back to ctf_bufopen is still positive: the new call site in ctf_track_enumerator does not need to change. (The existing tests for this reliably detect when I get it wrong. I know, because they did.) libctf/ * ctf-hash.c (ctf_dynhash_insert): Negate return value. (ctf_dynhash_insert_type): Set de-negated error on the dict: return negated error. * ctf-open.c (init_static_types_internal): Adapt to this change.
2024-07-31libctf, include: add ctf_dict_set_flag: less enum dup checking by defaultNick Alcock8-45/+181
The recent change to detect duplicate enum values and return ECTF_DUPLICATE when found turns out to perturb a great many callers. In particular, the pahole-created kernel BTF has the same problem we historically did, and gleefully emits duplicated enum constants in profusion. Handling the resulting duplicate errors from BTF -> CTF converters reasonably is unreasonably difficult (it amounts to forcing them to skip some types or reimplement the deduplicator). So let's step back a bit. What we care about mostly is that the deduplicator treat enums with conflicting enumeration constants as conflicting types: programs that want to look up enumeration constant -> value mappings using the new APIs to do so might well want the same checks to apply to any ctf_add_* operations they carry out (and since they're *using* the new APIs, added at the same time as this restriction was imposed, there is likely to be no negative consequence of this). So we want some way to allow processes that know about duplicate detection to opt into it, while allowing everyone else to stay clear of it: but we want ctf_link to get this behaviour even if its caller has opted out. So add a new concept to the API: dict-wide CTF flags, set via ctf_dict_set_flag, obtained via ctf_dict_get_flag. They are not bitflags but simple arbitrary integers and an on/off value, stored in an unspecified manner (the one current flag, we translate into an LCTF_* flag value in the internal ctf_dict ctf_flags word). If you pass in an invalid flag or value you get a new ECTF_BADFLAG error, so the caller can easily tell whether flags added in future are valid with a particular libctf or not. We check this flag in ctf_add_enumerator, and set it around the link (including on child per-CU dicts). The newish enumerator-iteration test is souped up to check the semantics of the flag as well. The fact that the flag can be set and unset at any time has curious consequences. You can unset the flag, insert a pile of duplicates, then set it and expect the new duplicates to be detected, not only by ctf_add_enumerator but also by ctf_lookup_enumerator. This means we now have to maintain the ctf_names and conflicting_enums enum-duplication tracking as new enums are added, not purely as the dict is opened. Move that code out of init_static_types_internal and into a new ctf_track_enumerator function that addition can also call. (None of this affects the file format or serialization machinery, which has to be able to handle duplicate enumeration constants no matter what.) include/ * ctf-api.h (CTF_ERRORS) [ECTF_BADFLAG]: New. (ECTF_NERR): Update. (CTF_STRICT_NO_DUP_ENUMERATORS): New flag. (ctf_dict_set_flag): New function. (ctf_dict_get_flag): Likewise. libctf/ * ctf-impl.h (LCTF_STRICT_NO_DUP_ENUMERATORS): New flag. (ctf_track_enumerator): Declare. * ctf-dedup.c (ctf_dedup_emit_type): Set it. * ctf-link.c (ctf_create_per_cu): Likewise. (ctf_link_deduplicating_per_cu): Likewise. (ctf_link): Likewise. (ctf_link_write): Likewise. * ctf-subr.c (ctf_dict_set_flag): New function. (ctf_dict_get_flag): New function. * ctf-open.c (init_static_types_internal): Move enum tracking to... * ctf-create.c (ctf_track_enumerator): ... this new function. (ctf_add_enumerator): Call it. * libctf.ver: Add the new functions. * testsuite/libctf-lookup/enumerator-iteration.c: Test them.
2024-07-31libctf: link: remember to turn off the LCTF_LINKING flag after ctf_link_writeNick Alcock1-0/+4
We set this flag at the top of ctf_link_write (to tell ctf_serialize, way down under the archive file writing functions, to do the various link- time serialization things like symbol filtering and the like), but we never remember to clear it except on error. This is probably bad if you want to serialize the dict yourself directly in the future after linking it (which is... definitely a *possible* use of the API, if rather strange). libctf/ * ctf-link.c (ctf_link_write): Clear LCTF_LINKING before exit.
2024-07-31libctf: link: fix error handlingNick Alcock1-1/+1
We were calling the wrong error function if opening failed, causing leaks. libctf/ * ctf-link.c (ctf_link_deduplicating_per_cu): Fix error handling.
2024-07-31libctf, open: Fix enum error handling pathNick Alcock1-0/+1
This new error-handling path was not properly initializing the fp's errno. libctf/ * ctf-open.c (init_static_types_internal): Set errno properly.
2024-07-31libctf, subr: don't mix up errors and warningsNick Alcock1-2/+2
ctf_err_warn() was debug-logging warnings as if they were errors and vice versa. libctf/ * ctf-subr.c (ctf_err_warn): Fix debugging thinko.
2024-07-31libctf: fix dynset insertionNick Alcock1-1/+1
libctf's dynsets are a straight wrapper around libiberty hashtab, storing the key directly in the hashtab slot. However, we'd often like to be able to store 0 and 1 (HTAB_EMPTY_ENTRY and HTAB_DELETED_ENTRY) in there, so we move them out of the way and replace them with huge unlikely values instead. Unfortunately we failed to do this replacement in one place, so insertion of 0 or 1 ended up misinforming the hashtab machinery that an entry was empty or deleted when it wasn't. libctf/ * ctf-hash.c (ctf_dynset_insert): Call key_to_internal properly.
2024-07-31libctf: dedup: tiny tweaksNick Alcock1-4/+3
Drop an unnecessary variable, and fix a buggy comment. No effect on generated code. libctf/ * ctf-dedup.c (ctf_dedup_detect_name_ambiguity): Drop unnecessary variable. (ctf_dedup_rwalk_output_mapping): Fix comment.
2024-07-31libctf: fix CTF dict compressionNick Alcock5-42/+179
Commit 483546ce4f3 ("libctf: make ctf_serialize() actually serialize") accidentally broke dict compression. There were two bugs: - ctf_arc_write_one_ctf was still making its own decision about whether to compress the dict via direct ctf_size comparison, which is unfortunate because now that it no longer calls ctf_serialize itself, ctf_size is always zero when it does this: it should let the writing functions decide on the threshold, which they contain code to do which is simply not used for lack of one trivial wrapper to write to an fd and also provide a compression threshold - ctf_write_mem, the function underlying all writing as of the commit above, was calling zlib's compressBound and avoiding compression if this returned a value larger than the input. Unfortunately compressBound does not do a trial compression and determine whether the result is compressible: it just adds zlib header sizes to the value passed in, so our test would *always* have concluded that the value was incompressible! Avoid by simply always compressing if the raw size is larger than the threshold: zlib is quite clever enough to avoid actually compressing if the data is incompressible. Add a testcase for this. libctf/ * ctf-impl.h (ctf_write_thresholded): New... * ctf-serialize.c (ctf_write_thresholded): ... defined here, a wrapper around... (ctf_write_mem): ... this. Don't check compressibility. (ctf_compress_write): Reimplement as a ctf_write_thresholded wrapper. (ctf_write): Likewise. * ctf-archive.c (arc_write_one_ctf): Just call ctf_write_thresholded rather than trying to work out whether to compress. * testsuite/libctf-writable/ctf-compressed.*: New test.
2024-07-31libctf: fix linking of non-root-visible typesNick Alcock3-2/+132
If you deduplicate non-root-visible types, the resulting type should still be non-root-visible! We were promoting all such types to root-visible, and re-demoting them only if their names collided (which might happen on cu-mapped links if multiple compilation units with conflicting types are fused into one child dict). This "worked" before now, in that linking at least didn't fail (if you don't mind having your non-root flag value destroyed if you're adding non-root-visible types), but now that conflicting enumerators cause their containing enums to become conflicted (enums which might have *different names*), this caused the linker to crash when it hit two enumerators with conflicting values. Not testable in ld because cu-mapped links are not exposed to ld, but can be tested via direct creation of libraries and calls to ctf_link directly. (This also tests the ctf_dump non-root type printout, which before now was untested.) libctf/ * ctf-dedup.c (ctf_dedup_emit_type): Non-root-visible input types should be emitted as non-root-visible output types. * testsuite/libctf-writable/ctf-nonroot-linking.c: New test. * testsuite/libctf-writable/ctf-nonroot-linking.lk: New test.
2024-07-31libctf, dump: correctly dump non-root-visible typesNick Alcock1-1/+1
The flag test when dumping non-root-visible tyeps was doubly wrong: the flags word is a *bitfield* containing CTF_ADD_ROOT as one possible value, so needs | and & testing, not just ==, and CTF_ADD_NONROOT is 0, so cannot be tested for this way: one must check for the non-presence of CTF_ADD_ROOT. libctf/ * ctf-dump.c (ctf_dump_format_type): Fix non-root flag test.
2024-07-31libctf, string: split the movable refs out of the ref listNick Alcock2-20/+47
In commit 149ce5c263616e65 we introduced the concept of "movable" refs, which are refs that can be moved in batches, to let us maintain valid ref lists even when adding refs to blocks of memory that can be realloced (which is any type containing a vlen which can expand, like names contained within enum or struct members). Movable refs need a backpointer to the movable refs dynhash for this dict; since non-movable refs are very common, we tried to save memory by having a slightly bigger struct for moveable refs with a backpointer in it, and casting appropriately, indicating which sort of ref we were dealing with via a flag on the atom. Unfortunately this doesn't work reliably, because you can perfectly well have a string ("foo", say) which has both non-movable refs (say, an external symbol and a variable name) and movable refs (say, a structure member name) to the same atom. Indicate which struct we're dealing with with an atom flag and suddenly you're casting a ctf_str_atom_ref to a ctf_str_atom_ref_movable (which is bigger) and dereferencing random memory off the end of it and interpreting it as a backpointer to the movable refs dynhash. This is unlikely to work well. So bite the bullet and split refs into two separate lists, one for movable refs, one for immovable refs. It means some annoying code duplication, but there's not very much of it, and it means we can keep the movable refs hashtab (which in turn means we don't have to do linear searches to find all relevant refs when moving refs, which in turn means that structure/union/enum member additions remain amortized O(n) time, not O(n^2). Callers can now purge movable and non-movable refs independently of each other. We don't use this yet, but a use is coming. libctf/ * ctf-impl.h (CTF_STR_ATOM_MOVABLE): Delete. (struct ctf_str_atom) [csa_movable_refs]: New. (struct ctf_dict): Adjust comment. (ctf_str_purge_refs): Add MOVABLE arg. * ctf-string.c (ctf_str_purge_movable_atom_refs): Split out of... (ctf_str_purge_atom_refs): ... this. (ctf_str_free_atom): Call it. (ctf_str_purge_one_atom_refs): Likewise. (aref_create): Adjust accordingly. (ctf_str_move_refs): Likewise. (ctf_str_remove_ref): Remove movable refs too, including deleting the ref from ctf_str_movable_refs. (ctf_str_purge_refs): Add MOVABLE arg. (ctf_str_update_refs): Update movable refs. (ctf_str_write_strtab): Check, and purge, movable refs.
2024-07-31libctf, dedup: drop unnecessary arg from ctf_dedup()Nick Alcock3-36/+30
The PARENTS arg is carefully passed down through all the layers of hash functions and then never used for anything. (In the distant past it was used for cycle detection, but the algorithm eventually committed doesn't need to do cycle detection...) The PARENTS arg is still used by ctf_dedup_emit(), but even there we can loosen the requirements and state that you can just leave entries corresponding to dicts with no parents at zero (which will be useful in an upcoming commit). libctf/ * ctf-dedup.c (ctf_dedup_hash_type): Drop PARENTS arg. (ctf_dedup_rhash_type): Likewise. (ctf_dedup): Likewise. (ctf_dedup_emit_struct_members): Mention what you can do to PARENTS entries for parent dicts. * ctf-impl.h (ctf_dedup): Adjust accordingly. * ctf-link.c (ctf_link_deduplicating_per_cu): Likewise. (ctf_link_deduplicating): Likewise.
2024-07-31libctf: we do in fact support foreign-endian old versionsNick Alcock1-8/+2
The worry that caused this to not be supported was because we don't bother endian-flipping version-related fields before checking them. But they're all unsigned chars anyway, and don't need any flipping at all. This should be supported and should already work. Enable it. libctf/ * ctf-open.c (ctf_bufopen): Don't prohibit foreign-endian upgrades.
2024-07-20Add markers for 2.43 branch/releaseNick Clifton1-0/+4
2024-06-20Revert "Remove LIBINTL_DEP"Alan Modra2-2/+5
This reverts commit e874cbd3879843a83e4bcc4b54cd7107387b1df6. The patch was wrong. LIBINTL_DEP is needed with an in-tree gettext.
2024-06-20Remove LIBINTL_DEPAlan Modra2-5/+2
The intl directory in the source no longer exists. LIBINTL_DEP is thus always empty. Remove references to it. config/ * gettext-sister.m4: Don't AC_SUBST LIBINTL_DEP. bfd/ * Makefile.in: Regenerate. * configure: Regenerate. binutils/ * Makefile.am (*_DEPENDENCIES): Remove LIBINTL_DEP. * Makefile.in: Regenerate. * configure: Regenerate. gas/ * Makefile.am (as_new_DEPENDENCIES): Remove LIBINTL_DEP. * Makefile.in: Regenerate. * configure: Regenerate. gdb/ * Makefile.in (INTL_DEPS): Don't set or reference. * configure: Regenerate. gdbserver/ * Makefile.in (INTL_DEPS): Don't set or reference. gdbsupport/ * Makefile.in: Regenerate. * configure: Regenerate. gold/ * Makefile.am (deps_var): Remove LIBINTL_DEP. (incremental_dump_DEPENDENCIES, dwp_DEPENDENCIES): Likewise. * Makefile.in: Regenerate. * configure: Regenerate. * testsuite/Makefile.am (DEPENDENCIES): Remove LIBINTL_DEP. * testsuite/Makefile.in: Regenerate. gprof/ * Makefile.am (gprof_DEPENDENCIES): Remove LIBINTL_DEP. * Makefile.in: Regenerate. * configure: Regenerate. ld/ * Makefile.am (ld_new_DEPENDENCIES): Remove LIBINTL_DEP. * Makefile.in: Regenerate. * configure: Regenerate. libctf/ * Makefile.in: Regenerate. * configure: Regenerate. opcodes/ * configure.ac (BUILD_LIBS): Remove LIBINTL. (BUILD_LIB_DEPS): Remove LIBINTL_DEP. * Makefile.in: Regenerate. * configure: Regenerate.
2024-06-19libctf: fix testsuite bugs revealed by -WallNick Alcock4-13/+9
Most of these are harmless, but some of the type confusions and especially a missing ctf_strerror() on an error path were actual bugs that could have resulted in test failures crashing rather than printing an error message. libctf/ * testsuite/libctf-lookup/enumerator-iteration.c: Fix type confusion, signedness confusion and a missing ctf_errmsg(). * testsuite/libctf-regression/libctf-repeat-cu-main.c: Return 0 from the test function. * testsuite/libctf-regression/open-error-free.c: Fix signedness confusion. * testsuite/libctf-regression/zrewrite.c: Remove unused label.
2024-06-18libctf, include: new functions for looking up enumeratorsNick Alcock8-9/+482
Three new functions for looking up the enum type containing a given enumeration constant, and optionally that constant's value. The simplest, ctf_lookup_enumerator, looks up a root-visible enumerator by name in one dict: if the dict contains multiple such constants (which is possible for dicts created by older versions of the libctf deduplicator), ECTF_DUPLICATE is returned. The next simplest, ctf_lookup_enumerator_next, is an iterator which returns all enumerators with a given name in a given dict, whether root-visible or not. The most elaborate, ctf_arc_lookup_enumerator_next, finds all enumerators with a given name across all dicts in an entire CTF archive, whether root-visible or not, starting looking in the shared parent dict; opened dicts are cached (as with all other ctf_arc_*lookup functions) so that repeated use does not incur repeated opening costs. All three of these return enumerator values as int64_t: unfortunately, API compatibility concerns prevent us from doing the same with the other older enum-related functions, which all return enumerator constant values as ints. We may be forced to add symbol-versioning compatibility aliases that fix the other functions in due course, bumping the soname for platforms that do not support such things. ctf_arc_lookup_enumerator_next is implemented as a nested ctf_archive_next iterator, and inside that, a nested ctf_lookup_enumerator_next iterator within each dict. To aid in this, add support to ctf_next_t iterators for iterators that are implemented in terms of two simultaneous nested iterators at once. (It has always been possible for callers to use as many nested or semi-overlapping ctf_next_t iterators as they need, which is one of the advantages of this style over the _iter style that calls a function for each thing iterated over: the iterator change here permits *ctf_next_t iterators themselves* to be implemented by iterating using multiple other iterators as part of their internal operation, transparently to the caller.) Also add a testcase that tests all these functions (which is fairly easy because ctf_arc_lookup_enumerator_next is implemented in terms of ctf_lookup_enumerator_next) in addition to enumeration addition in ctf_open()ed dicts, ctf_add_enumerator duplicate enumerator addition, and conflicting enumerator constant deduplication. include/ * ctf-api.h (ctf_lookup_enumerator): New. (ctf_lookup_enumerator_next): Likewise. (ctf_arc_lookup_enumerator_next): Likewise. libctf/ * libctf.ver: Add them. * ctf-impl.h (ctf_next_t) <ctn_next_inner>: New. * ctf-util.c (ctf_next_copy): Copy it. (ctf_next_destroy): Destroy it. * ctf-lookup.c (ctf_lookup_enumerator): New. (ctf_lookup_enumerator_next): New. * ctf-archive.c (ctf_arc_lookup_enumerator_next): New. * testsuite/libctf-lookup/enumerator-iteration.*: New test. * testsuite/libctf-lookup/enum-ctf-2.c: New test CTF, used by the above.
2024-06-18libctf: make the ctf_next ctn_fp non-constNick Alcock1-3/+5
This was always an error, because the ctn_fp routinely has errors set on it, which is not something you can (or should) do to a const object. libctf/ * ctf-impl.h (ctf_next_) <cu.ctn_fp>: Make non-const.
2024-06-18libctf: prohibit addition of enums with overlapping enumerator constantsNick Alcock4-21/+163
libctf has long prohibited addition of enums with overlapping constants in a single enum, but now that we are properly considering enums with overlapping constants to be conflciting types, we can go further and prohibit addition of enumeration constants to a dict if they already exist in any enum in that dict: the same rules as C itself. We do this in a fashion vaguely similar to what we just did in the deduplicator, by considering enumeration constants as identifiers and adding them to the core type/identifier namespace, ctf_dict_t.ctf_names. This is a little fiddly, because we do not want to prohibit opening of existing dicts into which the deduplicator has stuffed enums with overlapping constants! We just want to prohibit the addition of *new* enumerators that violate that rule. Even then, it's fine to add overlapping enumerator constants as long as at least one of them is in a non-root type. (This is essential for proper deduplicator operation in cu-mapped mode, where multiple compilation units can be smashed into one dict, with conflicting types marked as hidden: these types may well contain overlapping enumerators.) So, at open time, keep track of all enums observed, then do a third pass through the enums alone, adding each enumerator either to the ctf_names table as a mapping from the enumerator name to the enum it is part of (if not already present), or to a new ctf_conflicting_enums hashtable that tracks observed duplicates. (The latter is not used yet, but will be soon.) (We need to do a third pass because it's quite possible to have an enum containing an enumerator FOO followed by a type FOO: since they're processed in order, the enumerator would be processed before the type, and at that stage it seems nonconflicting. The easiest fix is to run through the enumerators after all type names are interned.) At ctf_add_enumerator time, if the enumerator to which we are adding a type is root-visible, check for an already-present name and error out if found, then intern the new name in the ctf_names table as is done at open time. (We retain the existing code which scans the enum itself for duplicates because it is still an error to add an enumerator twice to a non-root-visible enum type; but we only need to do this if the enum is non-root-visible, so the cost of enum addition is reduced.) Tested in an upcoming commit. libctf/ * ctf-impl.h (ctf_dict_t) <ctf_names>: Augment comment. <ctf_conflicting_enums>: New. (ctf_dynset_elements): New. * ctf-hash.c (ctf_dynset_elements): Implement it. * ctf-open.c (init_static_types): Split body into... (init_static_types_internal): ... here. Count enumerators; keep track of observed enums in pass 2; populate ctf_names and ctf_conflicting_enums with enumerators in a third pass. (ctf_dict_close): Free ctf_conflicting_enums. * ctf-create.c (ctf_add_enumerator): Prohibit addition of duplicate enumerators in root-visible enum types. include/ * ctf-api.h (CTF_ADD_NONROOT): Describe what non-rootness means for enumeration constants. (ctf_add_enumerator): The name is not a misnomer. We now require that enumerators have unique names. Document the non-rootness of enumerators.
2024-06-18libctf: suppress spurious failure of malloc-counting tests under valgrindNick Alcock5-2/+23
The libctf-regression/open-error-free.c test works by interposing malloc and counting mallocs and frees across libctf operations. This only works under suitably-interposable mallocs on systems supporting dlsym (RTLD_NEXT, ...), so its operation is restricted to glibc systems for now, but also it interacts badly with valgrind, which interposes malloc itself. Detect a running valgrind and skip the test. Add new facilities allowing libctf lookup tests to declare themselves unsupported, by printing "UNSUPPORTED: " and then some meaningful message instead of their normal output. libctf/ * configure.ac: Check for <valgrind/valgrind.h>. * config.h.in: Regenerate. * configure: Likewise. * testsuite/lib/ctf-lib.exp (run_lookup_test): Add support for UNSUPPORTED tests. * testsuite/libctf-regression/open-error-free.c: When running under valgrind, this test is unsupported.
2024-06-18libctf: fix dict leak on archive-wide symbol lookup error pathNick Alcock1-0/+1
If a lookup fails for a reason unrelated to a lack of type data for this symbol, we return with an error; but we fail to close the dict we opened most recently, which is leaked. libctf/ * ctf-archive.c (ctf_arc_lookup_sym_or_name): Close dict.
2024-06-18libctf: don't leak enums if ctf_add_type failsNick Alcock1-1/+6
If ctf_add_type failed in the middle of enumerator addition, the destination would end up containing the source enum type and some but not all of its enumerator constants. Use snapshots to roll back the enum addition as a whole if this happens. Before now, it's been pretty unlikely, but in an upcoming commit we will ban addition of enumerators that already exist in a given dict, making failure of ctf_add_enumerator and thus of this part of ctf_add_type much more likely. libctf/ * ctf-create.c (ctf_add_type_internal): Roll back if enum or enumerator addition fails.
2024-06-18libctf: dedup: enums with overlapping enumerators are conflictingNick Alcock2-7/+38
The CTF deduplicator was not considering enumerators inside enum types to be things that caused type conflicts, so if the following two TUs were linked together, you would end up with the following in the resulting dict: 1.c: enum foo { A, B }; 2.c: enum bar { A, B }; linked: enum foo { A, B }; enum bar { A, B }; This does work -- but it's not something that's valid C, and the general point of the shared dict is that it is something that you could potentially get from any valid C TU. So consider such types to be conflicting, but obviously don't consider actually identical enums to be conflicting, even though they too have (all) their identifiers in common. This involves surprisingly little code. The deduplicator detects conflicting types by counting types in a hash table of hash tables: decorated identifier -> (type hash -> count) where the COUNT is the number of times a given hash has been observed: any name with more than one hash associated with it is considered conflicting (the count is used to identify the most common such name for promotion to the shared dict). Before now, those identifiers were all the identifiers of types (possibly decorated with their namespace on the front for enumerator identifiers), but we can equally well put *enumeration constant names* in there, undecorated like the identifiers of types in the global namespace, with the type hash being the hash of each enum containing that enumerator. The existing conflicting-type-detection code will then accurately identify distinct enums with enumeration constants in common. The enum that contains the most commonly-appearing enumerators will be promoted to the shared dict. libctf/ * ctf-impl.h (ctf_dedup_t) <cd_name_counts>: Extend comment. * ctf-dedup.c (ctf_dedup_count_name): New, split out of... (ctf_dedup_populate_mappings): ... here. Call it for all * enumeration constants in an enum as well as types. ld/ * testsuite/ld-ctf/enum-3.c: New test CTF. * testsuite/ld-ctf/enum-4.c: Likewise. * testsuite/ld-ctf/overlapping-enums.d: New test. * testsuite/ld-ctf/overlapping-enums-2.d: Likewise.
2024-06-18libctf: doc: fix ctf_stype_t typedef string in specNick Alcock1-1/+1
2024-06-18libctf: strtab corruption when strings are added to ctf_open()ed dictsNick Alcock1-0/+2
ctf_str_add_ref and ctf_str_add_movable_ref take a string and are supposed to return a strtab offset. These offsets are "provisional": the ref mechanism records the address of the location in which the ref is stored and modifies it when the strtab is finally written out. Provisional refs in new dicts start at 0 and go up via strlen() as new refs are added: this is fine, because the strtab is empty and none of these values will overlap any existing string offsets (since there are none). Unfortunately, when a dict is ctf_open()ed, we fail to set the initial provisional strtab offset to a higher value than any existing string offset: it starts at zero again! It's a shame that we already *have* strings at those offsets... This is all fixed up once the string is reserialized, but if you look up newly-added strings before serialization, you get corrupted partial string results from the existing ctf_open()ed dict. Observed (and thus regtested) by an upcoming test (in this patch series). Exposed by the recently-introduced series that permits modification of ctf_open()ed dicts, which has not been released anywhere. Before that, any attempt to do such things would fail with ECTF_RDONLY. libctf/ * ctf-string.c (ctf_str_create_atoms): Initialize ctf_str_prov_offset.
2024-06-12PR 31882 libctf: test suite incorrect format specifiersA. Wilcox2-2/+2
2024-06-10autoupdate: add square brackets around arguments of AC_INITMatthieu Longo1-1/+1
https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.72/autoconf.html#index-AC_005fINIT-2