libctf, include: find types of symbols by name

The existing ctf_lookup_by_symbol and ctf_arc_lookup_symbol functions
suffice to look up the types of symbols if the caller already has a
symbol number.  But the caller often doesn't have one of those and only
knows the name of the symbol: also, in object files, the caller might
not have a useful symbol number in any sense (and neither does libctf:
the 'symbol number' we use in that case literally starts at 0 for the
lexicographically first-sorted symbol in the symtypetab and counts those
symbols, so it corresponds to nothing useful).

This means that even though object files have a symtypetab (generated by
the compiler or by ld -r), the only way we can look up anything in it is
to iterate over all symbols in turn with ctf_symbol_next until we find
the one we want.

This is unhelpful and pointlessly inefficient.

So add a pair of functions to look up symbols by name in a dict and in a
whole archive: ctf_lookup_by_symbol_name and ctf_arc_lookup_symbol_name.
These are identical to the existing functions except that they take
symbol names rather than symbol numbers.

To avoid insane repetition, we do some refactoring in the process, so
that both ctf_lookup_by_symbol and ctf_arc_lookup_symbol turn into thin
wrappers around internal functions that do both lookup by symbol index
and lookup by name.  This massively reduces code duplication because
even the existing lookup-by-index stuff wants to use a name sometimes
(when looking up in indexed sections), and the new lookup-by-name stuff
has to turn it into an index sometimes (when looking up in non-indexed
sections): doing it this way lets us share most of that.

The actual name->index lookup is done by ctf_lookup_symbol_idx.  We do
not anticipate this lookup to be as heavily used as ld.so symbol lookup
by many orders of magnitude, so using the ELF symbol hashes would
probably take more time to read them than is saved by using the hashes,
and it adds a lot of complexity.  Instead, do a linear search for the
symbol name, caching all the name -> index mappings as we go, so that
future searches are likely to hit in the cache.  To avoid having to
repeat this search over and over in a CTF archive when
ctf_arc_lookup_symbol_name is used, have cached archive lookups (the
sort done by ctf_arc_lookup_symbol* and the ctf_archive_next iterator)
pick out the first dict they cache in a given archive and store it in a
new ctf_archive field, ctfi_crossdict_cache.  This can be used to store
cross-dictionary cached state that depends on things like the ELF symbol
table rather than the contents of any one dict.  ctf_lookup_symbol_idx
then caches its name->index mappings in the dictionary named in the
crossdict cache, if any, so that ctf_lookup_symbol_idx in other dicts
in the same archive benefit from the previous linear search, and the
symtab only needs to be scanned at most once.

(Note that if you call ctf_lookup_by_symbol_name in one specific dict,
and then follow it with a ctf_arc_lookup_symbol_name, the former will
not use the crossdict cache because it's only populated by the dict
opens in ctf_arc_lookup_symbol_name. This is harmless except for a small
one-off waste of memory and time: it's only a cache, after all.  We can
fix this later by using the archive caching machinery more
aggressively.)

In ctf-archive, we do similar things, turning ctf_arc_lookup_symbol into
a wrapper around a new function that does both index -> ID and name ->
ID lookups across all dicts in an archive.  We add a new
ctfi_symnamedicts cache that maps symbol names to the ctf_dict_t * that
it was found in (so that linear searches for symbols don't need to be
repeated): but we also *remove* a cache, the ctfi_syms cache that was
memoizing the actual ctf_id_t returned from every call to
ctf_arc_lookup_symbol.  This is pointless: all it saves is one call to
ctf_lookup_by_symbol, and that's basically an array lookup and nothing
more so isn't worth caching.  (Equally, given that symbol -> index
mappings are cached by ctf_lookup_by_symbol_name, those calls are nearly
free after the first call, so there's no point caching the ctf_id_t in
that case either.)

We fix up one test that was doing manual symbol lookup to use
ctf_arc_lookup_symbol instead, and enhance it to check that the caching
layer is not totally broken: we also add a new test to do lookups in a
.o file, and another to do lookups in an archive with conflicted types
and make sure that sort of multi-dict lookup is actually working.

include/ChangeLog
2021-02-17  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-api.h (ctf_arc_lookup_symbol_name): New.
	(ctf_lookup_by_symbol_name): Likewise.

libctf/ChangeLog
2021-02-17  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-impl.h (ctf_dict_t) <ctf_symhash>: New.
	<ctf_symhash_latest>: Likewise.
	(struct ctf_archive_internal) <ctfi_crossdict_cache>: New.
	<ctfi_symnamedicts>: New.
	<ctfi_syms>: Remove.
	(ctf_lookup_symbol_name): Remove.
	* ctf-lookup.c (ctf_lookup_symbol_name): Propagate errors from
	parent properly.  Make static.
	(ctf_lookup_symbol_idx): New, linear search for the symbol name,
	cached in the crossdict cache's ctf_symhash (if available), or
	this dict's (otherwise).
	(ctf_try_lookup_indexed): Allow the symname to be passed in.
	(ctf_lookup_by_symbol): Turn into a wrapper around...
	(ctf_lookup_by_sym_or_name): ... this, supporting name lookup too,
	using ctf_lookup_symbol_idx in non-writable dicts.  Special-case
	name lookup in dynamic dicts without reported symbols, which have
	no symtab or dynsymidx but where name lookup should still work.
	(ctf_lookup_by_symbol_name): New, another wrapper.
	* ctf-archive.c (enosym): Note that this is present in
	ctfi_symnamedicts too.
	(ctf_arc_close): Adjust for removal of ctfi_syms.  Free the
	ctfi_symnamedicts.
	(ctf_arc_flush_caches): Likewise.
	(ctf_dict_open_cached): Memoize the first cached dict in the
	crossdict cache.
	(ctf_arc_lookup_symbol): Turn into a wrapper around...
	(ctf_arc_lookup_sym_or_name): ... this.  No longer cache
	ctf_id_t lookups: just call ctf_lookup_by_symbol as needed (but
	still cache the dicts those lookups succeed in).  Add
	lookup-by-name support, with dicts of successful lookups cached in
	ctfi_symnamedicts.  Refactor the caching code a bit.
	(ctf_arc_lookup_symbol_name): New, another wrapper.
	* ctf-open.c (ctf_dict_close): Free the ctf_symhash.
	* libctf.ver (LIBCTF_1.2): New version.  Add
	ctf_lookup_by_symbol_name, ctf_arc_lookup_symbol_name.
	* testsuite/libctf-lookup/enum-symbol.c (main): Use
	ctf_arc_lookup_symbol rather than looking up the name ourselves.
	Fish it out repeatedly, to make sure that symbol caching isn't
	broken.
	(symidx_64): Remove.
	(symidx_32): Remove.
	* testsuite/libctf-lookup/enum-symbol-obj.lk: Test symbol lookup
	in an unlinked object file (indexed symtypetab sections only).
	* testsuite/libctf-writable/symtypetab-nonlinker-writeout.c
	(try_maybe_reporting): Check symbol types via
	ctf_lookup_by_symbol_name as well as ctf_symbol_next.
	* testsuite/libctf-lookup/conflicting-type-syms.*: New test of
	lookups in a multi-dict archive.

1 files changed, 4 insertions, 2 deletions

diff --git a/libctf/ctf-impl.h b/libctf/ctf-impl.h
index 0a508eb..a6e1da5 100644
--- a/libctf/ctf-impl.h
+++ b/libctf/ctf-impl.h
@@ -382,6 +382,8 @@ struct ctf_dict
   ctf_sect_t ctf_symtab;	    /* Symbol table from object file.  */
   ctf_sect_t ctf_strtab;	    /* String table from object file.  */
   int ctf_symsect_little_endian;    /* Endianness of the ctf_symtab.  */
+  ctf_dynhash_t *ctf_symhash;       /* (partial) hash, symsect name -> idx. */
+  size_t ctf_symhash_latest;	    /* Amount of symsect scanned so far.  */
   ctf_dynhash_t *ctf_prov_strtab;   /* Maps provisional-strtab offsets
 				       to names.  */
   ctf_dynhash_t *ctf_syn_ext_strtab; /* Maps ext-strtab offsets to names.  */
@@ -508,8 +510,9 @@ struct ctf_archive_internal
   ctf_dict_t *ctfi_dict;
   struct ctf_archive *ctfi_archive;
   ctf_dynhash_t *ctfi_dicts;	  /* Dicts we have opened and cached.  */
+  ctf_dict_t *ctfi_crossdict_cache; /* Cross-dict caching.  */
   ctf_dict_t **ctfi_symdicts;	  /* Array of index -> ctf_dict_t *.  */
-  ctf_id_t *ctfi_syms;		  /* Array of index -> ctf_id_t.  */
+  ctf_dynhash_t *ctfi_symnamedicts; /* Hash of name -> ctf_dict_t *.  */
   ctf_sect_t ctfi_symsect;
   int ctfi_symsect_little_endian; /* -1 for unknown / do not set.  */
   ctf_sect_t ctfi_strsect;
@@ -784,7 +787,6 @@ extern ctf_link_sym_t *ctf_elf32_to_link_sym (ctf_dict_t *fp, ctf_link_sym_t *ds
 					      const Elf32_Sym *src, uint32_t symidx);
 extern ctf_link_sym_t *ctf_elf64_to_link_sym (ctf_dict_t *fp, ctf_link_sym_t *dst,
 					      const Elf64_Sym *src, uint32_t symidx);
-extern const char *ctf_lookup_symbol_name (ctf_dict_t *fp, unsigned long symidx);
 
 /* Variables, all underscore-prepended. */