libctf: symbol type linking support

This adds facilities to write out the function info and data object
sections, which efficiently map from entries in the symbol table to
types.  The write-side code is entirely new: the read-side code was
merely significantly changed and support for indexed tables added
(pointed to by the no-longer-unused cth_objtidxoff and cth_funcidxoff
header fields).

With this in place, you can use ctf_lookup_by_symbol to look up the
types of symbols of function and object type (and, as before, you can
use ctf_lookup_variable to look up types of file-scope variables not
present in the symbol table, as long as you know their name: but
variables that are also data objects are now found in the data object
section instead.)

(Compatible) file format change:

The CTF spec has always said that the function info section looks much
like the CTF_K_FUNCTIONs in the type section: an info word (including an
argument count) followed by a return type and N argument types. This
format is suboptimal: it means function symbols cannot be deduplicated
and it causes a lot of ugly code duplication in libctf.  But
conveniently the compiler has never emitted this!  Because it has always
emitted a rather different format that libctf has never accepted, we can
be sure that there are no instances of this function info section in the
wild, and can freely change its format without compatibility concerns or
a file format version bump.  (And since it has never been emitted in any
code that generated any older file format version, either, we need keep
no code to read the format as specified at all!)

So the function info section is now specified as an array of uint32_t,
exactly like the object data section: each entry is a type ID in the
type section which must be of kind CTF_K_FUNCTION, the prototype of
this function.

This allows function types to be deduplicated and also correctly encodes
the fact that all functions declared in C really are types available to
the program: so they should be stored in the type section like all other
types.  (In format v4, we will be able to represent the types of static
functions as well, but that really does require a file format change.)

We introduce a new header flag, CTF_F_NEWFUNCINFO, which is set if the
new function info format is in use.  A sufficiently new compiler will
always set this flag.  New libctf will always set this flag: old libctf
will refuse to open any CTF dicts that have this flag set.  If the flag
is not set on a dict being read in, new libctf will disregard the
function info section.  Format v4 will remove this flag (or, rather, the
flag has no meaning there and the bit position may be recycled for some
other purpose).

New API:

Symbol addition:
  ctf_add_func_sym: Add a symbol with a given name and type.  The
                    type must be of kind CTF_K_FUNCTION (a function
                    pointer).  Internally this adds a name -> type
                    mapping to the ctf_funchash in the ctf_dict.
  ctf_add_objt_sym: Add a symbol with a given name and type.  The type
                    kind can be anything, including function pointers.
		    This adds to ctf_objthash.

These both treat symbols as name -> type mappings: the linker associates
symbol names with symbol indexes via the ctf_link_shuffle_syms callback,
which sets up the ctf_dynsyms/ctf_dynsymidx/ctf_dynsymmax fields in the
ctf_dict.  Repeated relinks can add more symbols.

Variables that are also exposed as symbols are removed from the variable
section at serialization time.

CTF symbol type sections which have enough pads, defined by
CTF_INDEX_PAD_THRESHOLD (whether because they are in dicts with symbols
where most types are unknown, or in archive where most types are defined
in some child or parent dict, not in this specific dict) are sorted by
name rather than symidx and accompanied by an index which associates
each symbol type entry with a name: the existing ctf_lookup_by_symbol
will map symbol indexes to symbol names and look the names up in the
index automatically.  (This is currently ELF-symbol-table-dependent, but
there is almost nothing specific to ELF in here and we can add support
for other symbol table formats easily).

The compiler also uses index sections to communicate the contents of
object file symbol tables without relying on any specific ordering of
symbols: it doesn't need to sort them, and libctf will detect an
unsorted index section via the absence of the new CTF_F_IDXSORTED header
flag, and sort it if needed.

Iteration:
  ctf_symbol_next: Iterator which returns the types and names of symbols
                   one by one, either for function or data symbols.

This does not require any sorting: the ctf_link machinery uses it to
pull in all the compiler-provided symbols cheaply, but it is not
restricted to that use.

(Compatible) changes in API:
  ctf_lookup_by_symbol: can now be called for object and function
                        symbols: never returns ECTF_NOTDATA (which is
			now not thrown by anything, but is kept for
                        compatibility and because it is a plausible
                        error that we might start throwing again at some
                        later date).

Internally we also have changes to the ctf-string functionality so that
"external" strings (those where we track a string -> offset mapping, but
only write out an offset) can be consulted via the usual means
(ctf_strptr) before the strtab is written out.  This is important
because ctf_link_add_linker_symbol can now be handed symbols named via
strtab offsets, and ctf_link_shuffle_syms must figure out their actual
names by looking in the external symtab we have just been fed by the
ctf_link_add_strtab callback, long before that strtab is written out.

include/ChangeLog
2020-11-20  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-api.h (ctf_symbol_next): New.
	(ctf_add_objt_sym): Likewise.
	(ctf_add_func_sym): Likewise.
	* ctf.h: Document new function info section format.
	(CTF_F_NEWFUNCINFO): New.
	(CTF_F_IDXSORTED): New.
	(CTF_F_MAX): Adjust accordingly.

libctf/ChangeLog
2020-11-20  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-impl.h (CTF_INDEX_PAD_THRESHOLD): New.
	(_libctf_nonnull_): Likewise.
	(ctf_in_flight_dynsym_t): New.
	(ctf_dict_t) <ctf_funcidx_names>: Likewise.
	<ctf_objtidx_names>: Likewise.
	<ctf_nfuncidx>: Likewise.
	<ctf_nobjtidx>: Likewise.
	<ctf_funcidx_sxlate>: Likewise.
	<ctf_objtidx_sxlate>: Likewise.
	<ctf_objthash>: Likewise.
	<ctf_funchash>: Likewise.
	<ctf_dynsyms>: Likewise.
	<ctf_dynsymidx>: Likewise.
	<ctf_dynsymmax>: Likewise.
	<ctf_in_flight_dynsym>: Likewise.
	(struct ctf_next) <u.ctn_next>: Likewise.
	(ctf_symtab_skippable): New prototype.
	(ctf_add_funcobjt_sym): Likewise.
	(ctf_dynhash_sort_by_name): Likewise.
	(ctf_sym_to_elf64): Rename to...
	(ctf_elf32_to_link_sym): ... this, and...
	(ctf_elf64_to_link_sym): ... this.
	* ctf-open.c (init_symtab): Check for lack of CTF_F_NEWFUNCINFO
	flag, and presence of index sections.  Refactor out
	ctf_symtab_skippable and ctf_elf*_to_link_sym, and use them.  Use
	ctf_link_sym_t, not Elf64_Sym.  Skip initializing objt or func
	sxlate sections if corresponding index section is present.  Adjust
	for new func info section format.
	(ctf_bufopen_internal): Add ctf_err_warn to corrupt-file error
	handling.  Report incorrect-length index sections.  Always do an
	init_symtab, even if there is no symtab section (there may be index
	sections still).
	(flip_objts): Adjust comment: func and objt sections are actually
	identical in structure now, no need to caveat.
	(ctf_dict_close):  Free newly-added data structures.
	* ctf-create.c (ctf_create): Initialize them.
	(ctf_symtab_skippable): New, refactored out of
	init_symtab, with st_nameidx_set check added.
	(ctf_add_funcobjt_sym): New, add a function or object symbol to the
	ctf_objthash or ctf_funchash, by name.
	(ctf_add_objt_sym): Call it.
	(ctf_add_func_sym): Likewise.
	(symtypetab_delete_nonstatic_vars): New, delete vars also present as
	data objects.
	(CTF_SYMTYPETAB_EMIT_FUNCTION): New flag to symtypetab emitters:
	this is a function emission, not a data object emission.
	(CTF_SYMTYPETAB_EMIT_PAD): New flag to symtypetab emitters: emit
	pads for symbols with no type (only set for unindexed sections).
	(CTF_SYMTYPETAB_FORCE_INDEXED): New flag to symtypetab emitters:
	always emit indexed.
	(symtypetab_density): New, figure out section sizes.
	(emit_symtypetab): New, emit a symtypetab.
	(emit_symtypetab_index): New, emit a symtypetab index.
	(ctf_serialize): Call them, emitting suitably sorted symtypetab
	sections and indexes.  Set suitable header flags.  Copy over new
	fields.
	* ctf-hash.c (ctf_dynhash_sort_by_name): New, used to impose an
	order on symtypetab index sections.
	* ctf-link.c (ctf_add_type_mapping): Delete erroneous comment
	relating to code that was never committed.
	(ctf_link_one_variable): Improve variable name.
	(check_sym): New, symtypetab analogue of check_variable.
	(ctf_link_deduplicating_one_symtypetab): New.
	(ctf_link_deduplicating_syms): Likewise.
	(ctf_link_deduplicating): Call them.
	(ctf_link_deduplicating_per_cu): Note that we don't call them in
	this case (yet).
	(ctf_link_add_strtab): Set the error on the fp correctly.
	(ctf_link_add_linker_symbol): New (no longer a do-nothing stub), add
	a linker symbol to the in-flight list.
	(ctf_link_shuffle_syms): New (no longer a do-nothing stub), turn the
	in-flight list into a mapping we can use, now its names are
	resolvable in the external strtab.
	* ctf-string.c (ctf_str_rollback_atom): Don't roll back atoms with
	external strtab offsets.
	(ctf_str_rollback): Adjust comment.
	(ctf_str_write_strtab): Migrate ctf_syn_ext_strtab population from
	writeout time...
	(ctf_str_add_external): ... to string addition time.
	* ctf-lookup.c (ctf_lookup_var_key_t): Rename to...
	(ctf_lookup_idx_key_t): ... this, now we use it for syms too.
	<clik_names>: New member, a name table.
	(ctf_lookup_var): Adjust accordingly.
	(ctf_lookup_variable): Likewise.
	(ctf_lookup_by_id): Shuffle further up in the file.
	(ctf_symidx_sort_arg_cb): New, callback for...
	(sort_symidx_by_name): ... this new function to sort a symidx
	found to be unsorted (likely originating from the compiler).
	(ctf_symidx_sort): New, sort a symidx.
	(ctf_lookup_symbol_name): Support dynamic symbols with indexes
	provided by the linker.  Use ctf_link_sym_t, not Elf64_Sym.
	Check the parent if a child lookup fails.
	(ctf_lookup_by_symbol): Likewise.  Work for function symbols too.
	(ctf_symbol_next): New, iterate over symbols with types (without
	sorting).
	(ctf_lookup_idx_name): New, bsearch for symbol names in indexes.
	(ctf_try_lookup_indexed): New, attempt an indexed lookup.
	(ctf_func_info): Reimplement in terms of ctf_lookup_by_symbol.
	(ctf_func_args): Likewise.
	(ctf_get_dict): Move...
	* ctf-types.c (ctf_get_dict): ... here.
	* ctf-util.c (ctf_sym_to_elf64): Re-express as...
	(ctf_elf64_to_link_sym): ... this.  Add new st_symidx field, and
	st_nameidx_set (always 0, so st_nameidx can be ignored).  Look in
	the ELF strtab for names.
	(ctf_elf32_to_link_sym): Likewise, for Elf32_Sym.
	(ctf_next_destroy): Destroy ctf_next_t.u.ctn_next if need be.
	* libctf.ver: Add ctf_symbol_next, ctf_add_objt_sym and
	ctf_add_func_sym.

1 files changed, 440 insertions, 132 deletions

diff --git a/libctf/ctf-lookup.c b/libctf/ctf-lookup.c
index 5682bf0..a862b40 100644
--- a/libctf/ctf-lookup.c
+++ b/libctf/ctf-lookup.c
@@ -20,6 +20,7 @@
 #include <ctf-impl.h>
 #include <elf.h>
 #include <string.h>
+#include <assert.h>
 
 /* Compare the given input string and length against a table of known C storage
    qualifier keywords.  We just ignore these in ctf_lookup_by_name, below.  To
@@ -192,21 +193,66 @@ err:
   return CTF_ERR;
 }
 
-typedef struct ctf_lookup_var_key
+/* Return the pointer to the internal CTF type data corresponding to the
+   given type ID.  If the ID is invalid, the function returns NULL.
+   This function is not exported outside of the library.  */
+
+const ctf_type_t *
+ctf_lookup_by_id (ctf_dict_t **fpp, ctf_id_t type)
 {
-  ctf_dict_t *clvk_fp;
-  const char *clvk_name;
-} ctf_lookup_var_key_t;
+  ctf_dict_t *fp = *fpp;	/* Caller passes in starting CTF dict.  */
+  ctf_id_t idx;
+
+  if ((fp = ctf_get_dict (fp, type)) == NULL)
+    {
+      (void) ctf_set_errno (*fpp, ECTF_NOPARENT);
+      return NULL;
+    }
+
+  /* If this dict is writable, check for a dynamic type.  */
+
+  if (fp->ctf_flags & LCTF_RDWR)
+    {
+      ctf_dtdef_t *dtd;
+
+      if ((dtd = ctf_dynamic_type (fp, type)) != NULL)
+	{
+	  *fpp = fp;
+	  return &dtd->dtd_data;
+	}
+      (void) ctf_set_errno (*fpp, ECTF_BADID);
+      return NULL;
+    }
+
+  /* Check for a type in the static portion.  */
+
+  idx = LCTF_TYPE_TO_INDEX (fp, type);
+  if (idx > 0 && (unsigned long) idx <= fp->ctf_typemax)
+    {
+      *fpp = fp;		/* Function returns ending CTF dict.  */
+      return (LCTF_INDEX_TO_TYPEPTR (fp, idx));
+    }
+
+  (void) ctf_set_errno (*fpp, ECTF_BADID);
+  return NULL;
+}
+
+typedef struct ctf_lookup_idx_key
+{
+  ctf_dict_t *clik_fp;
+  const char *clik_name;
+  uint32_t *clik_names;
+} ctf_lookup_idx_key_t;
 
 /* A bsearch function for variable names.  */
 
 static int
-ctf_lookup_var (const void *key_, const void *memb_)
+ctf_lookup_var (const void *key_, const void *lookup_)
 {
-  const ctf_lookup_var_key_t *key = key_;
-  const ctf_varent_t *memb = memb_;
+  const ctf_lookup_idx_key_t *key = key_;
+  const ctf_varent_t *lookup = lookup_;
 
-  return (strcmp (key->clvk_name, ctf_strptr (key->clvk_fp, memb->ctv_name)));
+  return (strcmp (key->clik_name, ctf_strptr (key->clik_fp, lookup->ctv_name)));
 }
 
 /* Given a variable name, return the type of the variable with that name.  */
@@ -215,7 +261,7 @@ ctf_id_t
 ctf_lookup_variable (ctf_dict_t *fp, const char *name)
 {
   ctf_varent_t *ent;
-  ctf_lookup_var_key_t key = { fp, name };
+  ctf_lookup_idx_key_t key = { fp, name, NULL };
 
   /* This array is sorted, so we can bsearch for it.  */
 
@@ -233,189 +279,457 @@ ctf_lookup_variable (ctf_dict_t *fp, const char *name)
   return ent->ctv_type;
 }
 
-/* Given a symbol table index, return the name of that symbol from the secondary
-   string table, or the null string (never NULL).  */
+typedef struct ctf_symidx_sort_arg_cb
+{
+  ctf_dict_t *fp;
+  uint32_t *names;
+} ctf_symidx_sort_arg_cb_t;
+
+static int
+sort_symidx_by_name (const void *one_, const void *two_, void *arg_)
+{
+  const uint32_t *one = one_;
+  const uint32_t *two = two_;
+  ctf_symidx_sort_arg_cb_t *arg = arg_;
+
+  return (strcmp (ctf_strptr (arg->fp, arg->names[*one]),
+		  ctf_strptr (arg->fp, arg->names[*two])));
+}
+
+/* Sort a symbol index section by name.  Takes a 1:1 mapping of names to the
+   corresponding symbol table.  Returns a lexicographically sorted array of idx
+   indexes (and thus, of indexes into the corresponding func info / data object
+   section).  */
+
+static uint32_t *
+ctf_symidx_sort (ctf_dict_t *fp, uint32_t *idx, size_t *nidx,
+			 size_t len)
+{
+  uint32_t *sorted;
+  size_t i;
+
+  if ((sorted = malloc (len)) == NULL)
+    {
+      ctf_set_errno (fp, ENOMEM);
+      return NULL;
+    }
+
+  *nidx = len / sizeof (uint32_t);
+  for (i = 0; i < *nidx; i++)
+    sorted[i] = i;
+
+  if (!(fp->ctf_header->cth_flags & CTF_F_IDXSORTED))
+    {
+      ctf_symidx_sort_arg_cb_t arg = { fp, idx };
+      ctf_dprintf ("Index section unsorted: sorting.");
+      ctf_qsort_r (sorted, *nidx, sizeof (uint32_t), sort_symidx_by_name, &arg);
+      fp->ctf_header->cth_flags |= CTF_F_IDXSORTED;
+    }
+
+  return sorted;
+}
+
+/* Given a symbol index, return the name of that symbol from the table provided
+   by ctf_link_shuffle_syms, or failing that from the secondary string table, or
+   the null string.  */
 const char *
 ctf_lookup_symbol_name (ctf_dict_t *fp, unsigned long symidx)
 {
   const ctf_sect_t *sp = &fp->ctf_symtab;
-  Elf64_Sym sym, *gsp;
+  ctf_link_sym_t sym;
+  int err;
 
-  if (sp->cts_data == NULL)
+  if (fp->ctf_dynsymidx)
     {
-      ctf_set_errno (fp, ECTF_NOSYMTAB);
-      return _CTF_NULLSTR;
+      err = EINVAL;
+      if (symidx > fp->ctf_dynsymmax)
+	goto try_parent;
+
+      ctf_link_sym_t *symp = fp->ctf_dynsymidx[symidx];
+
+      if (!symp)
+	goto try_parent;
+
+      return symp->st_name;
     }
 
+  err = ECTF_NOSYMTAB;
+  if (sp->cts_data == NULL)
+    goto try_parent;
+
   if (symidx >= fp->ctf_nsyms)
+    goto try_parent;
+
+  switch (sp->cts_entsize)
     {
-      ctf_set_errno (fp, EINVAL);
+    case sizeof (Elf64_Sym):
+      {
+	const Elf64_Sym *symp = (Elf64_Sym *) sp->cts_data + symidx;
+	ctf_elf64_to_link_sym (fp, &sym, symp, symidx);
+      }
+      break;
+    case sizeof (Elf32_Sym):
+      {
+	const Elf32_Sym *symp = (Elf32_Sym *) sp->cts_data + symidx;
+	ctf_elf32_to_link_sym (fp, &sym, symp, symidx);
+      }
+      break;
+    default:
+      ctf_set_errno (fp, ECTF_SYMTAB);
       return _CTF_NULLSTR;
     }
 
-  if (sp->cts_entsize == sizeof (Elf32_Sym))
-    {
-      const Elf32_Sym *symp = (Elf32_Sym *) sp->cts_data + symidx;
-      gsp = ctf_sym_to_elf64 (symp, &sym);
-    }
-  else
-      gsp = (Elf64_Sym *) sp->cts_data + symidx;
+  assert (!sym.st_nameidx_set);
 
-  if (gsp->st_name < fp->ctf_str[CTF_STRTAB_1].cts_len)
-    return (const char *) fp->ctf_str[CTF_STRTAB_1].cts_strs + gsp->st_name;
+  return sym.st_name;
 
-  return _CTF_NULLSTR;
+ try_parent:
+  if (fp->ctf_parent)
+    return ctf_lookup_symbol_name (fp->ctf_parent, symidx);
+  else
+    {
+      ctf_set_errno (fp, err);
+      return _CTF_NULLSTR;
+    }
 }
 
-/* Given a symbol table index, return the type of the data object described
-   by the corresponding entry in the symbol table.  */
+/* Iterate over all symbols with types: if FUNC, function symbols, otherwise,
+   data symbols.  The name argument is not optional.  The return order is
+   arbitrary, though is likely to be in symbol index or name order.  You can
+   change the value of 'functions' in the middle of iteration over non-dynamic
+   dicts, but doing so on dynamic dicts will fail.  (This is probably not very
+   useful, but there is no reason to prohibit it.)  */
 
 ctf_id_t
-ctf_lookup_by_symbol (ctf_dict_t *fp, unsigned long symidx)
+ctf_symbol_next (ctf_dict_t *fp, ctf_next_t **it, const char **name,
+		 int functions)
 {
-  const ctf_sect_t *sp = &fp->ctf_symtab;
-  ctf_id_t type;
+  ctf_id_t sym;
+  ctf_next_t *i = *it;
+  int err;
 
-  if (sp->cts_data == NULL)
-    return (ctf_set_errno (fp, ECTF_NOSYMTAB));
+  if (!i)
+    {
+      if ((i = ctf_next_create ()) == NULL)
+	return ctf_set_errno (fp, ENOMEM);
 
-  if (symidx >= fp->ctf_nsyms)
-    return (ctf_set_errno (fp, EINVAL));
+      i->cu.ctn_fp = fp;
+      i->ctn_iter_fun = (void (*) (void)) ctf_symbol_next;
+      i->ctn_n = 0;
+      *it = i;
+    }
+
+  if ((void (*) (void)) ctf_symbol_next != i->ctn_iter_fun)
+    return (ctf_set_errno (fp, ECTF_NEXT_WRONGFUN));
 
-  if (sp->cts_entsize == sizeof (Elf32_Sym))
+  if (fp != i->cu.ctn_fp)
+    return (ctf_set_errno (fp, ECTF_NEXT_WRONGFP));
+
+  /* We intentionally use raw access, not ctf_lookup_by_symbol, to avoid
+     incurring additional sorting cost for unsorted symtypetabs coming from the
+     compiler, to allow ctf_symbol_next to work in the absence of a symtab, and
+     finally because it's easier to work out what the name of each symbol is if
+     we do that.  */
+
+  if (fp->ctf_flags & LCTF_RDWR)
     {
-      const Elf32_Sym *symp = (Elf32_Sym *) sp->cts_data + symidx;
-      if (ELF32_ST_TYPE (symp->st_info) != STT_OBJECT)
-	return (ctf_set_errno (fp, ECTF_NOTDATA));
+      ctf_dynhash_t *dynh = functions ? fp->ctf_funchash : fp->ctf_objthash;
+      void *dyn_name = NULL, *dyn_value = NULL;
+
+      if (!dynh)
+	{
+	  ctf_next_destroy (i);
+	  return (ctf_set_errno (fp, ECTF_NEXT_END));
+	}
+
+      err = ctf_dynhash_next (dynh, &i->u.ctn_next, &dyn_name, &dyn_value);
+      /* This covers errors and also end-of-iteration.  */
+      if (err != 0)
+	{
+	  ctf_next_destroy (i);
+	  *it = NULL;
+	  return ctf_set_errno (fp, err);
+	}
+
+      *name = dyn_name;
+      sym = (ctf_id_t) (uintptr_t) dyn_value;
     }
-  else
+  else if ((!functions && fp->ctf_objtidx_names) ||
+	   (functions && fp->ctf_funcidx_names))
     {
-      const Elf64_Sym *symp = (Elf64_Sym *) sp->cts_data + symidx;
-      if (ELF64_ST_TYPE (symp->st_info) != STT_OBJECT)
-	return (ctf_set_errno (fp, ECTF_NOTDATA));
+      ctf_header_t *hp = fp->ctf_header;
+      uint32_t *idx = functions ? fp->ctf_funcidx_names : fp->ctf_objtidx_names;
+      uint32_t *tab;
+      size_t len;
+
+      if (functions)
+	{
+	  len = (hp->cth_varoff - hp->cth_funcidxoff) / sizeof (uint32_t);
+	  tab = (uint32_t *) (fp->ctf_buf + hp->cth_funcoff);
+	}
+      else
+	{
+	  len = (hp->cth_funcidxoff - hp->cth_objtidxoff) / sizeof (uint32_t);
+	  tab = (uint32_t *) (fp->ctf_buf + hp->cth_objtoff);
+	}
+
+      do
+	{
+	  if (i->ctn_n >= len)
+	    goto end;
+
+	  *name = ctf_strptr (fp, idx[i->ctn_n]);
+	  sym = tab[i->ctn_n++];
+	} while (sym == -1u || sym == 0);
     }
+  else
+    {
+      /* Skip over pads in ctf_xslate, padding for typeless symbols in the
+	 symtypetab itself, and symbols in the wrong table.  */
+      for (; i->ctn_n < fp->ctf_nsyms; i->ctn_n++)
+	{
+	  ctf_header_t *hp = fp->ctf_header;
 
-  if (fp->ctf_sxlate[symidx] == -1u)
-    return (ctf_set_errno (fp, ECTF_NOTYPEDAT));
+	  if (fp->ctf_sxlate[i->ctn_n] == -1u)
+	    continue;
 
-  type = *(uint32_t *) ((uintptr_t) fp->ctf_buf + fp->ctf_sxlate[symidx]);
-  if (type == 0)
-    return (ctf_set_errno (fp, ECTF_NOTYPEDAT));
+	  sym = *(uint32_t *) ((uintptr_t) fp->ctf_buf + fp->ctf_sxlate[i->ctn_n]);
 
-  return type;
+	  if (sym == 0)
+	    continue;
+
+	  if (functions)
+	    {
+	      if (fp->ctf_sxlate[i->ctn_n] >= hp->cth_funcoff
+		  && fp->ctf_sxlate[i->ctn_n] < hp->cth_objtidxoff)
+		break;
+	    }
+	  else
+	    {
+	      if (fp->ctf_sxlate[i->ctn_n] >= hp->cth_objtoff
+		  && fp->ctf_sxlate[i->ctn_n] < hp->cth_funcoff)
+		break;
+	    }
+	}
+
+      if (i->ctn_n >= fp->ctf_nsyms)
+	goto end;
+
+      *name = ctf_lookup_symbol_name (fp, i->ctn_n++);
+    }
+
+  return sym;
+
+ end:
+  ctf_next_destroy (i);
+  *it = NULL;
+  return (ctf_set_errno (fp, ECTF_NEXT_END));
 }
 
-/* Return the native dict of a given type: if called on a child and the
-   type is in the parent, return the parent.  Needed if you plan to access
-   the type directly, without using the API.  */
-ctf_dict_t *
-ctf_get_dict (ctf_dict_t *fp, ctf_id_t type)
+/* A bsearch function for function and object index names.  */
+
+static int
+ctf_lookup_idx_name (const void *key_, const void *idx_)
 {
-    if ((fp->ctf_flags & LCTF_CHILD) && LCTF_TYPE_ISPARENT (fp, type))
-      return fp->ctf_parent;
+  const ctf_lookup_idx_key_t *key = key_;
+  const uint32_t *idx = idx_;
 
-    return fp;
+  return (strcmp (key->clik_name, ctf_strptr (key->clik_fp, key->clik_names[*idx])));
 }
 
-/* Return the pointer to the internal CTF type data corresponding to the
-   given type ID.  If the ID is invalid, the function returns NULL.
-   This function is not exported outside of the library.  */
+/* Given a symbol number, look up that symbol in the function or object
+   index table (which must exist).  Return 0 if not found there (or pad).  */
 
-const ctf_type_t *
-ctf_lookup_by_id (ctf_dict_t **fpp, ctf_id_t type)
+static ctf_id_t
+ctf_try_lookup_indexed (ctf_dict_t *fp, unsigned long symidx, int is_function)
 {
-  ctf_dict_t *fp = *fpp;	/* Caller passes in starting CTF dict.  */
-  ctf_id_t idx;
+  const char *symname = ctf_lookup_symbol_name (fp, symidx);
+  struct ctf_header *hp = fp->ctf_header;
+  uint32_t *symtypetab;
+  uint32_t *names;
+  uint32_t *sxlate;
+  size_t nidx;
 
-  if ((fp = ctf_get_dict (fp, type)) == NULL)
-    {
-      (void) ctf_set_errno (*fpp, ECTF_NOPARENT);
-      return NULL;
-    }
+  ctf_dprintf ("Looking up type of object with symtab idx %lx (%s) in "
+	       "indexed symtypetab\n", symidx, symname);
 
-  /* If this dict is writable, check for a dynamic type.  */
+  if (symname[0] == '\0')
+    return -1;					/* errno is set for us.  */
 
-  if (fp->ctf_flags & LCTF_RDWR)
+  if (is_function)
     {
-      ctf_dtdef_t *dtd;
-
-      if ((dtd = ctf_dynamic_type (fp, type)) != NULL)
+      if (!fp->ctf_funcidx_sxlate)
 	{
-	  *fpp = fp;
-	  return &dtd->dtd_data;
+	  if ((fp->ctf_funcidx_sxlate
+	       = ctf_symidx_sort (fp, (uint32_t *)
+				  (fp->ctf_buf + hp->cth_funcidxoff),
+				  &fp->ctf_nfuncidx,
+				  hp->cth_varoff - hp->cth_funcidxoff))
+	      == NULL)
+	    {
+	      ctf_err_warn (fp, 0, 0, _("cannot sort function symidx"));
+	      return -1;				/* errno is set for us.  */
+	    }
 	}
-      (void) ctf_set_errno (*fpp, ECTF_BADID);
-      return NULL;
+      symtypetab = (uint32_t *) (fp->ctf_buf + hp->cth_funcoff);
+      sxlate = fp->ctf_funcidx_sxlate;
+      names = fp->ctf_funcidx_names;
+      nidx = fp->ctf_nfuncidx;
     }
+  else
+    {
+      if (!fp->ctf_objtidx_sxlate)
+	{
+	  if ((fp->ctf_objtidx_sxlate
+	       = ctf_symidx_sort (fp, (uint32_t *)
+				  (fp->ctf_buf + hp->cth_objtidxoff),
+				  &fp->ctf_nobjtidx,
+				  hp->cth_funcidxoff - hp->cth_objtidxoff))
+	      == NULL)
+	    {
+	      ctf_err_warn (fp, 0, 0, _("cannot sort object symidx"));
+	      return -1;				/* errno is set for us. */
+	    }
+	}
 
-  /* Check for a type in the static portion.  */
+      symtypetab = (uint32_t *) (fp->ctf_buf + hp->cth_objtoff);
+      sxlate = fp->ctf_objtidx_sxlate;
+      names = fp->ctf_objtidx_names;
+      nidx = fp->ctf_nobjtidx;
+    }
 
-  idx = LCTF_TYPE_TO_INDEX (fp, type);
-  if (idx > 0 && (unsigned long) idx <= fp->ctf_typemax)
+  ctf_lookup_idx_key_t key = { fp, symname, names };
+  uint32_t *idx;
+
+  idx = bsearch (&key, sxlate, nidx, sizeof (uint32_t), ctf_lookup_idx_name);
+
+  if (!idx)
     {
-      *fpp = fp;		/* Function returns ending CTF dict.  */
-      return (LCTF_INDEX_TO_TYPEPTR (fp, idx));
+      ctf_dprintf ("%s not found in idx\n", symname);
+      return 0;
     }
 
-  (void) ctf_set_errno (*fpp, ECTF_BADID);
-  return NULL;
+  /* Should be impossible, but be paranoid.  */
+  if ((idx - sxlate) > (ptrdiff_t) nidx)
+    return (ctf_set_errno (fp, ECTF_CORRUPT));
+
+  ctf_dprintf ("Symbol %lx (%s) is of type %x\n", symidx, symname,
+	       symtypetab[*idx]);
+  return symtypetab[*idx];
 }
 
-/* Given a symbol table index, return the info for the function described
-   by the corresponding entry in the symbol table.  */
+/* Given a symbol table index, return the type of the function or data object
+   described by the corresponding entry in the symbol table.  We can only return
+   symbols in read-only dicts and in dicts for which ctf_link_shuffle_syms has
+   been called to assign symbol indexes to symbol names.  */
 
-int
-ctf_func_info (ctf_dict_t *fp, unsigned long symidx, ctf_funcinfo_t *fip)
+ctf_id_t
+ctf_lookup_by_symbol (ctf_dict_t *fp, unsigned long symidx)
 {
   const ctf_sect_t *sp = &fp->ctf_symtab;
-  const uint32_t *dp;
-  uint32_t info, kind, n;
+  ctf_id_t type = 0;
+  int err = 0;
+
+  /* Shuffled dynsymidx present?  Use that.  */
+  if (fp->ctf_dynsymidx)
+    {
+      const ctf_link_sym_t *sym;
+
+      ctf_dprintf ("Looking up type of object with symtab idx %lx in "
+		   "writable dict symtypetab\n", symidx);
+
+      /* The dict must be dynamic.  */
+      if (!ctf_assert (fp, fp->ctf_flags & LCTF_RDWR))
+	return CTF_ERR;
+
+      err = EINVAL;
+      if (symidx > fp->ctf_dynsymmax)
+	goto try_parent;
+
+      sym = fp->ctf_dynsymidx[symidx];
+      err = ECTF_NOTYPEDAT;
+      if (!sym || (sym->st_shndx != STT_OBJECT && sym->st_shndx != STT_FUNC))
+	goto try_parent;
+
+      if (!ctf_assert (fp, !sym->st_nameidx_set))
+	return CTF_ERR;
+
+      if (fp->ctf_objthash == NULL
+	  || ((type = (ctf_id_t) (uintptr_t)
+	       ctf_dynhash_lookup (fp->ctf_objthash, sym->st_name)) == 0))
+	{
+	  if (fp->ctf_funchash == NULL
+	      || ((type = (ctf_id_t) (uintptr_t)
+		   ctf_dynhash_lookup (fp->ctf_funchash, sym->st_name)) == 0))
+	    goto try_parent;
+	}
+
+      return type;
+    }
 
+  err = ECTF_NOSYMTAB;
   if (sp->cts_data == NULL)
-    return (ctf_set_errno (fp, ECTF_NOSYMTAB));
+    goto try_parent;
 
+  /* This covers both out-of-range lookups and a dynamic dict which hasn't been
+     shuffled yet.  */
+  err = EINVAL;
   if (symidx >= fp->ctf_nsyms)
-    return (ctf_set_errno (fp, EINVAL));
+    goto try_parent;
 
-  if (sp->cts_entsize == sizeof (Elf32_Sym))
+  if (fp->ctf_objtidx_names)
     {
-      const Elf32_Sym *symp = (Elf32_Sym *) sp->cts_data + symidx;
-      if (ELF32_ST_TYPE (symp->st_info) != STT_FUNC)
-	return (ctf_set_errno (fp, ECTF_NOTFUNC));
+      if ((type = ctf_try_lookup_indexed (fp, symidx, 0)) == CTF_ERR)
+	return CTF_ERR;				/* errno is set for us.  */
     }
-  else
+  if (type == 0 && fp->ctf_funcidx_names)
     {
-      const Elf64_Sym *symp = (Elf64_Sym *) sp->cts_data + symidx;
-      if (ELF64_ST_TYPE (symp->st_info) != STT_FUNC)
-	return (ctf_set_errno (fp, ECTF_NOTFUNC));
+      if ((type = ctf_try_lookup_indexed (fp, symidx, 1)) == CTF_ERR)
+	return CTF_ERR;				/* errno is set for us.  */
     }
+  if (type != 0)
+    return type;
+
+  err = ECTF_NOTYPEDAT;
+  if (fp->ctf_objtidx_names && fp->ctf_funcidx_names)
+    goto try_parent;
+
+  /* Table must be nonindexed.  */
+
+  ctf_dprintf ("Looking up object type %lx in 1:1 dict symtypetab\n", symidx);
 
   if (fp->ctf_sxlate[symidx] == -1u)
-    return (ctf_set_errno (fp, ECTF_NOFUNCDAT));
+    goto try_parent;
+
+  type = *(uint32_t *) ((uintptr_t) fp->ctf_buf + fp->ctf_sxlate[symidx]);
 
-  dp = (uint32_t *) ((uintptr_t) fp->ctf_buf + fp->ctf_sxlate[symidx]);
+  if (type == 0)
+    goto try_parent;
 
-  info = *dp++;
-  kind = LCTF_INFO_KIND (fp, info);
-  n = LCTF_INFO_VLEN (fp, info);
+  return type;
+ try_parent:
+  if (fp->ctf_parent)
+    return ctf_lookup_by_symbol (fp->ctf_parent, symidx);
+  else
+    return (ctf_set_errno (fp, err));
+}
 
-  if (kind == CTF_K_UNKNOWN && n == 0)
-    return (ctf_set_errno (fp, ECTF_NOFUNCDAT));
+/* Given a symbol table index, return the info for the function described
+   by the corresponding entry in the symbol table, which may be a function
+   symbol or may be a data symbol that happens to be a function pointer.  */
 
-  if (kind != CTF_K_FUNCTION)
-    return (ctf_set_errno (fp, ECTF_CORRUPT));
+int
+ctf_func_info (ctf_dict_t *fp, unsigned long symidx, ctf_funcinfo_t *fip)
+{
+  ctf_id_t type;
 
-  fip->ctc_return = *dp++;
-  fip->ctc_argc = n;
-  fip->ctc_flags = 0;
+  if ((type = ctf_lookup_by_symbol (fp, symidx)) == CTF_ERR)
+    return -1;					/* errno is set for us.  */
 
-  if (n != 0 && dp[n - 1] == 0)
-    {
-      fip->ctc_flags |= CTF_FUNC_VARARG;
-      fip->ctc_argc--;
-    }
+  if (ctf_type_kind (fp, type) != CTF_K_FUNCTION)
+    return (ctf_set_errno (fp, ECTF_NOTFUNC));
 
-  return 0;
+  return ctf_func_type_info (fp, type, fip);
 }
 
 /* Given a symbol table index, return the arguments for the function described
@@ -423,21 +737,15 @@ ctf_func_info (ctf_dict_t *fp, unsigned long symidx, ctf_funcinfo_t *fip)
 
 int
 ctf_func_args (ctf_dict_t *fp, unsigned long symidx, uint32_t argc,
-	       ctf_id_t * argv)
+	       ctf_id_t *argv)
 {
-  const uint32_t *dp;
-  ctf_funcinfo_t f;
-
-  if (ctf_func_info (fp, symidx, &f) < 0)
-    return -1;			/* errno is set for us.  */
-
-  /* The argument data is two uint32_t's past the translation table
-     offset: one for the function info, and one for the return type. */
+  ctf_id_t type;
 
-  dp = (uint32_t *) ((uintptr_t) fp->ctf_buf + fp->ctf_sxlate[symidx]) + 2;
+  if ((type = ctf_lookup_by_symbol (fp, symidx)) == CTF_ERR)
+    return -1;					/* errno is set for us.  */
 
-  for (argc = MIN (argc, f.ctc_argc); argc != 0; argc--)
-    *argv++ = *dp++;
+  if (ctf_type_kind (fp, type) != CTF_K_FUNCTION)
+    return (ctf_set_errno (fp, ECTF_NOTFUNC));
 
-  return 0;
+  return ctf_func_type_args (fp, type, argc, argv);
 }