libctf: support getting strings from the ELF strtab

The CTF file format has always supported "external strtabs", which
internally are strtab offsets with their MSB on: such refs
get their strings from the strtab passed in at CTF file open time:
this is usually intended to be the ELF strtab, and that's what this
implementation is meant to support, though in theory the external
strtab could come from anywhere.

This commit adds support for these external strings in the ctf-string.c
strtab tracking layer.  It's quite easy: we just add a field csa_offset
to the atoms table that tracks all strings: this field tracks the offset
of the string in the ELF strtab (with its MSB already on, courtesy of a
new macro CTF_SET_STID), and adds a new function that sets the
csa_offset to the specified offset (plus MSB).  Then we just need to
avoid writing out strings to the internal strtab if they have csa_offset
set, and note that the internal strtab is shorter than it might
otherwise be.

(We could in theory save a little more time here by eschewing sorting
such strings, since we never actually write the strings out anywhere,
but that would mean storing them separately and it's just not worth the
complexity cost until profiling shows it's worth doing.)

We also have to go through a bit of extra effort at variable-sorting
time.  This was previously using direct references to the internal
strtab: it couldn't use ctf_strptr or ctf_strraw because the new strtab
is not yet ready to put in its usual field (in a ctf_file_t that hasn't
even been allocated yet at this stage): but now we're using the external
strtab, this will no longer do because it'll be looking things up in the
wrong strtab, with disastrous results.  Instead, pass the new internal
strtab in to a new ctf_strraw_explicit function which is just like
ctf_strraw except you can specify a ne winternal strtab to use.

But even now that it is using a new internal strtab, this is not quite
enough: it can't look up strings in the external strtab because ld
hasn't written it out yet, and when it does will write it straight to
disk.  Instead, when we write the internal strtab, note all the offset
-> string mappings that we have noted belong in the *external* strtab to
a new "synthetic external strtab" dynhash, ctf_syn_ext_strtab, and look
in there at ctf_strraw time if it is set.  This uses minimal extra
memory (because only strings in the external strtab that we actually use
are stored, and even those come straight out of the atoms table), but
let both variable sorting and name interning when ctf_bufopen is next
called work fine.  (This also means that we don't need to filter out
spurious ECTF_STRTAB warnings from ctf_bufopen but can pass them back to
the caller, once we wrap ctf_bufopen so that we have a new internal
variant of ctf_bufopen etc that we can pass the synthetic external
strtab to. That error has been filtered out since the days of Solaris
libctf, which didn't try to handle the problem of getting external
strtabs right at construction time at all.)

v3: add the synthetic strtab and all associated machinery.
v5: fix tabdamage.

include/
	* ctf.h (CTF_SET_STID): New.

libctf/
	* ctf-impl.h (ctf_str_atom_t) <csa_offset>: New field.
	(ctf_file_t) <ctf_syn_ext_strtab>: Likewise.
	(ctf_str_add_ref): Name the last arg.
	(ctf_str_add_external) New.
	(ctf_str_add_strraw_explicit): Likewise.
	(ctf_simple_open_internal): Likewise.
	(ctf_bufopen_internal): Likewise.

	* ctf-string.c (ctf_strraw_explicit): Split from...
	(ctf_strraw): ... here, with new support for ctf_syn_ext_strtab.
	(ctf_str_add_ref_internal): Return the atom, not the
	string.
	(ctf_str_add): Adjust accordingly.
	(ctf_str_add_ref): Likewise.  Move up in the file.
	(ctf_str_add_external): New: update the csa_offset.
	(ctf_str_count_strtab): Only account for strings with no csa_offset
	in the internal strtab length.
	(ctf_str_write_strtab): If the csa_offset is set, update the
	string's refs without writing the string out, and update the
	ctf_syn_ext_strtab.  Make OOM handling less ugly.
	* ctf-create.c (struct ctf_sort_var_arg_cb): New.
	(ctf_update): Handle failure to populate the strtab.  Pass in the
	new ctf_sort_var arg.  Adjust for ctf_syn_ext_strtab addition.
	Call ctf_simple_open_internal, not ctf_simple_open.
	(ctf_sort_var): Call ctf_strraw_explicit rather than looking up
	strings by hand.
	* ctf-hash.c (ctf_hash_insert_type): Likewise (but using
	ctf_strraw).  Adjust to diagnose ECTF_STRTAB nonetheless.
	* ctf-open.c (init_types): No longer filter out ECTF_STRTAB.
	(ctf_file_close): Destroy the ctf_syn_ext_strtab.
	(ctf_simple_open): Rename to, and reimplement as a wrapper around...
	(ctf_simple_open_internal): ... this new function, which calls
	ctf_bufopen_internal.
	(ctf_bufopen): Rename to, and reimplement as a wrapper around...
	(ctf_bufopen_internal): ... this new function, which sets
	ctf_syn_ext_strtab.

6 files changed, 261 insertions, 70 deletions

diff --git a/libctf/ChangeLog b/libctf/ChangeLog
index df52650..fbf765f 100644
--- a/libctf/ChangeLog
+++ b/libctf/ChangeLog
@@ -1,5 +1,44 @@
 2019-07-13  Nick Alcock  <nick.alcock@oracle.com>
 
+	* ctf-impl.h (ctf_str_atom_t) <csa_offset>: New field.
+	(ctf_file_t) <ctf_syn_ext_strtab>: Likewise.
+	(ctf_str_add_ref): Name the last arg.
+	(ctf_str_add_external) New.
+	(ctf_str_add_strraw_explicit): Likewise.
+	(ctf_simple_open_internal): Likewise.
+	(ctf_bufopen_internal): Likewise.
+
+	* ctf-string.c (ctf_strraw_explicit): Split from...
+	(ctf_strraw): ... here, with new support for ctf_syn_ext_strtab.
+	(ctf_str_add_ref_internal): Return the atom, not the
+	string.
+	(ctf_str_add): Adjust accordingly.
+	(ctf_str_add_ref): Likewise.  Move up in the file.
+	(ctf_str_add_external): New: update the csa_offset.
+	(ctf_str_count_strtab): Only account for strings with no csa_offset
+	in the internal strtab length.
+	(ctf_str_write_strtab): If the csa_offset is set, update the
+	string's refs without writing the string out, and update the
+	ctf_syn_ext_strtab.  Make OOM handling less ugly.
+	* ctf-create.c (struct ctf_sort_var_arg_cb): New.
+	(ctf_update): Handle failure to populate the strtab.  Pass in the
+	new ctf_sort_var arg.  Adjust for ctf_syn_ext_strtab addition.
+	Call ctf_simple_open_internal, not ctf_simple_open.
+	(ctf_sort_var): Call ctf_strraw_explicit rather than looking up
+	strings by hand.
+	* ctf-hash.c (ctf_hash_insert_type): Likewise (but using
+	ctf_strraw).  Adjust to diagnose ECTF_STRTAB nonetheless.
+	* ctf-open.c (init_types): No longer filter out ECTF_STRTAB.
+	(ctf_file_close): Destroy the ctf_syn_ext_strtab.
+	(ctf_simple_open): Rename to, and reimplement as a wrapper around...
+	(ctf_simple_open_internal): ... this new function, which calls
+	ctf_bufopen_internal.
+	(ctf_bufopen): Rename to, and reimplement as a wrapper around...
+	(ctf_bufopen_internal): ... this new function, which sets
+	ctf_syn_ext_strtab.
+
+2019-07-13  Nick Alcock  <nick.alcock@oracle.com>
+
 	* ctf_types.c (ctf_type_iter_all): New.
 
 2019-07-13  Nick Alcock  <nick.alcock@oracle.com>
diff --git a/libctf/ctf-create.c b/libctf/ctf-create.c
index cef2351..3e6a319 100644
--- a/libctf/ctf-create.c
+++ b/libctf/ctf-create.c
@@ -170,31 +170,37 @@ ctf_copy_emembers (ctf_file_t *fp, ctf_dtdef_t *dtd, unsigned char *t)
 
 /* Sort a newly-constructed static variable array.  */
 
+typedef struct ctf_sort_var_arg_cb
+{
+  ctf_file_t *fp;
+  ctf_strs_t *strtab;
+} ctf_sort_var_arg_cb_t;
+
 static int
-ctf_sort_var (const void *one_, const void *two_, void *strtab_)
+ctf_sort_var (const void *one_, const void *two_, void *arg_)
 {
   const ctf_varent_t *one = one_;
   const ctf_varent_t *two = two_;
-  const char *strtab = strtab_;
-  const char *n1 = strtab + CTF_NAME_OFFSET (one->ctv_name);
-  const char *n2 = strtab + CTF_NAME_OFFSET (two->ctv_name);
+  ctf_sort_var_arg_cb_t *arg = arg_;
 
-  return (strcmp (n1, n2));
+  return (strcmp (ctf_strraw_explicit (arg->fp, one->ctv_name, arg->strtab),
+		  ctf_strraw_explicit (arg->fp, two->ctv_name, arg->strtab)));
 }
 
 /* If the specified CTF container is writable and has been modified, reload this
    container with the updated type definitions.  In order to make this code and
    the rest of libctf as simple as possible, we perform updates by taking the
    dynamic type definitions and creating an in-memory CTF file containing the
-   definitions, and then call ctf_simple_open() on it.  This not only leverages
-   ctf_simple_open(), but also avoids having to bifurcate the rest of the library
-   code with different lookup paths for static and dynamic type definitions.  We
-   are therefore optimizing greatly for lookup over update, which we assume will
-   be an uncommon operation.  We perform one extra trick here for the benefit of
-   callers and to keep our code simple: ctf_simple_open() will return a new
-   ctf_file_t, but we want to keep the fp constant for the caller, so after
-   ctf_simple_open() returns, we use memcpy to swap the interior of the old and
-   new ctf_file_t's, and then free the old.  */
+   definitions, and then call ctf_simple_open_internal() on it.  This not only
+   leverages ctf_simple_open(), but also avoids having to bifurcate the rest of
+   the library code with different lookup paths for static and dynamic type
+   definitions.  We are therefore optimizing greatly for lookup over update,
+   which we assume will be an uncommon operation.  We perform one extra trick
+   here for the benefit of callers and to keep our code simple:
+   ctf_simple_open_internal() will return a new ctf_file_t, but we want to keep
+   the fp constant for the caller, so after ctf_simple_open_internal() returns,
+   we use memcpy to swap the interior of the old and new ctf_file_t's, and then
+   free the old.  */
 int
 ctf_update (ctf_file_t *fp)
 {
@@ -412,10 +418,17 @@ ctf_update (ctf_file_t *fp)
   strtab = ctf_str_write_strtab (fp);
   ctf_str_purge_refs (fp);
 
+  if (strtab.cts_strs == NULL)
+    {
+      ctf_free (buf);
+      return (ctf_set_errno (fp, EAGAIN));
+    }
+
   /* Now the string table is constructed, we can sort the buffer of
      ctf_varent_t's.  */
+  ctf_sort_var_arg_cb_t sort_var_arg = { fp, (ctf_strs_t *) &strtab };
   ctf_qsort_r (dvarents, nvars, sizeof (ctf_varent_t), ctf_sort_var,
-	       strtab.cts_strs);
+	       &sort_var_arg);
 
   if ((newbuf = ctf_realloc (fp, buf, buf_size + strtab.cts_len)) == NULL)
     {
@@ -433,8 +446,9 @@ ctf_update (ctf_file_t *fp)
   /* Finally, we are ready to ctf_simple_open() the new container.  If this
      is successful, we then switch nfp and fp and free the old container.  */
 
-  if ((nfp = ctf_simple_open ((char *) buf, buf_size, NULL, 0, 0, NULL,
-			      0, &err)) == NULL)
+  if ((nfp = ctf_simple_open_internal ((char *) buf, buf_size, NULL, 0,
+				       0, NULL, 0, fp->ctf_syn_ext_strtab,
+				       &err)) == NULL)
     {
       ctf_free (buf);
       return (ctf_set_errno (fp, err));
@@ -456,6 +470,7 @@ ctf_update (ctf_file_t *fp)
   nfp->ctf_dtoldid = fp->ctf_dtnextid - 1;
   nfp->ctf_snapshots = fp->ctf_snapshots + 1;
   nfp->ctf_specific = fp->ctf_specific;
+  nfp->ctf_syn_ext_strtab = fp->ctf_syn_ext_strtab;
 
   nfp->ctf_snapshot_lu = fp->ctf_snapshots;
 
@@ -465,6 +480,7 @@ ctf_update (ctf_file_t *fp)
   nfp->ctf_str_atoms = fp->ctf_str_atoms;
   fp->ctf_str_atoms = NULL;
   memset (&fp->ctf_dtdefs, 0, sizeof (ctf_list_t));
+  fp->ctf_syn_ext_strtab = NULL;
 
   fp->ctf_dvhash = NULL;
   memset (&fp->ctf_dvdefs, 0, sizeof (ctf_list_t));
diff --git a/libctf/ctf-hash.c b/libctf/ctf-hash.c
index 12bd6ef..3512d22 100644
--- a/libctf/ctf-hash.c
+++ b/libctf/ctf-hash.c
@@ -271,16 +271,18 @@ int
 ctf_hash_insert_type (ctf_hash_t *hp, ctf_file_t *fp, uint32_t type,
 		      uint32_t name)
 {
-  ctf_strs_t *ctsp = &fp->ctf_str[CTF_NAME_STID (name)];
-  const char *str = ctsp->cts_strs + CTF_NAME_OFFSET (name);
+  const char *str = ctf_strraw (fp, name);
 
   if (type == 0)
     return EINVAL;
 
-  if (ctsp->cts_strs == NULL)
+  if (str == NULL
+      && CTF_NAME_STID (name) == CTF_STRTAB_1
+      && fp->ctf_syn_ext_strtab == NULL
+      && fp->ctf_str[CTF_NAME_STID (name)].cts_strs == NULL)
     return ECTF_STRTAB;
 
-  if (ctsp->cts_len <= CTF_NAME_OFFSET (name))
+  if (str == NULL)
     return ECTF_BADNAME;
 
   if (str[0] == '\0')
diff --git a/libctf/ctf-impl.h b/libctf/ctf-impl.h
index 5b331cb..03c48cf 100644
--- a/libctf/ctf-impl.h
+++ b/libctf/ctf-impl.h
@@ -192,6 +192,7 @@ typedef struct ctf_str_atom
 {
   const char *csa_str;		/* Backpointer to string (hash key).  */
   ctf_list_t csa_refs;		/* This string's refs.  */
+  uint32_t csa_offset;		/* External strtab offset, if any.  */
   unsigned long csa_snapshot_id; /* Snapshot ID at time of creation.  */
 } ctf_str_atom_t;
 
@@ -222,6 +223,7 @@ struct ctf_file
   ctf_sect_t ctf_data;		    /* CTF data from object file.  */
   ctf_sect_t ctf_symtab;	    /* Symbol table from object file.  */
   ctf_sect_t ctf_strtab;	    /* String table from object file.  */
+  ctf_dynhash_t *ctf_syn_ext_strtab; /* Maps ext-strtab offsets to names.  */
   void *ctf_data_mmapped;	    /* CTF data we mmapped, to free later.  */
   size_t ctf_data_mmapped_len;	    /* Length of CTF data we mmapped.  */
   ctf_hash_t *ctf_structs;	    /* Hash table of struct types.  */
@@ -375,12 +377,15 @@ _libctf_printflike_ (2, 3)
 extern void ctf_decl_sprintf (ctf_decl_t *, const char *, ...);
 extern char *ctf_decl_buf (ctf_decl_t *cd);
 
-extern const char *ctf_strraw (ctf_file_t *, uint32_t);
 extern const char *ctf_strptr (ctf_file_t *, uint32_t);
+extern const char *ctf_strraw (ctf_file_t *, uint32_t);
+extern const char *ctf_strraw_explicit (ctf_file_t *, uint32_t,
+					ctf_strs_t *);
 extern int ctf_str_create_atoms (ctf_file_t *);
 extern void ctf_str_free_atoms (ctf_file_t *);
 extern const char *ctf_str_add (ctf_file_t *, const char *);
-extern const char *ctf_str_add_ref (ctf_file_t *, const char *, uint32_t *);
+extern const char *ctf_str_add_ref (ctf_file_t *, const char *, uint32_t *ref);
+extern const char *ctf_str_add_external (ctf_file_t *, const char *, uint32_t offset);
 extern void ctf_str_rollback (ctf_file_t *, ctf_snapshot_id_t);
 extern void ctf_str_purge_refs (ctf_file_t *);
 extern ctf_strs_writable_t ctf_str_write_strtab (ctf_file_t *);
@@ -391,6 +396,14 @@ extern void ctf_arc_close_internal (struct ctf_archive *);
 extern void *ctf_set_open_errno (int *, int);
 extern unsigned long ctf_set_errno (ctf_file_t *, int);
 
+extern ctf_file_t *ctf_simple_open_internal (const char *, size_t, const char *,
+					     size_t, size_t,
+					     const char *, size_t,
+					     ctf_dynhash_t *, int *);
+extern ctf_file_t *ctf_bufopen_internal (const ctf_sect_t *, const ctf_sect_t *,
+					 const ctf_sect_t *, ctf_dynhash_t *,
+					 int *);
+
 _libctf_malloc_
 extern void *ctf_mmap (size_t length, size_t offset, int fd);
 extern void ctf_munmap (void *, size_t);
diff --git a/libctf/ctf-open.c b/libctf/ctf-open.c
index c96bad7..f4179cb 100644
--- a/libctf/ctf-open.c
+++ b/libctf/ctf-open.c
@@ -785,7 +785,7 @@ init_types (ctf_file_t *fp, ctf_header_t *cth)
 	      err = ctf_hash_define_type (fp->ctf_names, fp,
 					  LCTF_INDEX_TO_TYPE (fp, id, child),
 					  tp->ctt_name);
-	      if (err != 0 && err != ECTF_STRTAB)
+	      if (err != 0)
 		return err;
 	    }
 	  break;
@@ -800,7 +800,7 @@ init_types (ctf_file_t *fp, ctf_header_t *cth)
 	  err = ctf_hash_insert_type (fp->ctf_names, fp,
 				      LCTF_INDEX_TO_TYPE (fp, id, child),
 				      tp->ctt_name);
-	  if (err != 0 && err != ECTF_STRTAB)
+	  if (err != 0)
 	    return err;
 	  break;
 
@@ -809,7 +809,7 @@ init_types (ctf_file_t *fp, ctf_header_t *cth)
 				      LCTF_INDEX_TO_TYPE (fp, id, child),
 				      tp->ctt_name);
 
-	  if (err != 0 && err != ECTF_STRTAB)
+	  if (err != 0)
 	    return err;
 
 	  if (size >= CTF_LSTRUCT_THRESH)
@@ -821,7 +821,7 @@ init_types (ctf_file_t *fp, ctf_header_t *cth)
 				      LCTF_INDEX_TO_TYPE (fp, id, child),
 				      tp->ctt_name);
 
-	  if (err != 0 && err != ECTF_STRTAB)
+	  if (err != 0)
 	    return err;
 
 	  if (size >= CTF_LSTRUCT_THRESH)
@@ -833,7 +833,7 @@ init_types (ctf_file_t *fp, ctf_header_t *cth)
 				      LCTF_INDEX_TO_TYPE (fp, id, child),
 				      tp->ctt_name);
 
-	  if (err != 0 && err != ECTF_STRTAB)
+	  if (err != 0)
 	    return err;
 	  break;
 
@@ -841,7 +841,7 @@ init_types (ctf_file_t *fp, ctf_header_t *cth)
 	  err = ctf_hash_insert_type (fp->ctf_names, fp,
 				      LCTF_INDEX_TO_TYPE (fp, id, child),
 				      tp->ctt_name);
-	  if (err != 0 && err != ECTF_STRTAB)
+	  if (err != 0)
 	    return err;
 	  break;
 
@@ -868,7 +868,7 @@ init_types (ctf_file_t *fp, ctf_header_t *cth)
 	      err = ctf_hash_insert_type (hp, fp,
 					  LCTF_INDEX_TO_TYPE (fp, id, child),
 					  tp->ctt_name);
-	      if (err != 0 && err != ECTF_STRTAB)
+	      if (err != 0)
 		return err;
 	    }
 	  break;
@@ -889,7 +889,7 @@ init_types (ctf_file_t *fp, ctf_header_t *cth)
 	  err = ctf_hash_insert_type (fp->ctf_names, fp,
 				      LCTF_INDEX_TO_TYPE (fp, id, child),
 				      tp->ctt_name);
-	  if (err != 0 && err != ECTF_STRTAB)
+	  if (err != 0)
 	    return err;
 	  break;
 	default:
@@ -1191,12 +1191,27 @@ flip_ctf (ctf_header_t *cth, unsigned char *buf)
 }
 
 /* Open a CTF file, mocking up a suitable ctf_sect.  */
+
 ctf_file_t *ctf_simple_open (const char *ctfsect, size_t ctfsect_size,
 			     const char *symsect, size_t symsect_size,
 			     size_t symsect_entsize,
 			     const char *strsect, size_t strsect_size,
 			     int *errp)
 {
+  return ctf_simple_open_internal (ctfsect, ctfsect_size, symsect, symsect_size,
+				   symsect_entsize, strsect, strsect_size, NULL,
+				   errp);
+}
+
+/* Open a CTF file, mocking up a suitable ctf_sect and overriding the external
+   strtab with a synthetic one.  */
+
+ctf_file_t *ctf_simple_open_internal (const char *ctfsect, size_t ctfsect_size,
+				      const char *symsect, size_t symsect_size,
+				      size_t symsect_entsize,
+				      const char *strsect, size_t strsect_size,
+				      ctf_dynhash_t *syn_strtab, int *errp)
+{
   ctf_sect_t skeleton;
 
   ctf_sect_t ctf_sect, sym_sect, str_sect;
@@ -1232,7 +1247,7 @@ ctf_file_t *ctf_simple_open (const char *ctfsect, size_t ctfsect_size,
       strsectp = &str_sect;
     }
 
-  return ctf_bufopen (ctfsectp, symsectp, strsectp, errp);
+  return ctf_bufopen_internal (ctfsectp, symsectp, strsectp, syn_strtab, errp);
 }
 
 /* Decode the specified CTF buffer and optional symbol table, and create a new
@@ -1244,6 +1259,16 @@ ctf_file_t *
 ctf_bufopen (const ctf_sect_t *ctfsect, const ctf_sect_t *symsect,
 	     const ctf_sect_t *strsect, int *errp)
 {
+  return ctf_bufopen_internal (ctfsect, symsect, strsect, NULL, errp);
+}
+
+/* Like ctf_bufopen, but overriding the external strtab with a synthetic one.  */
+
+ctf_file_t *
+ctf_bufopen_internal (const ctf_sect_t *ctfsect, const ctf_sect_t *symsect,
+		      const ctf_sect_t *strsect, ctf_dynhash_t *syn_strtab,
+		      int *errp)
+{
   const ctf_preamble_t *pp;
   size_t hdrsz = sizeof (ctf_header_t);
   ctf_header_t *hp;
@@ -1253,7 +1278,8 @@ ctf_bufopen (const ctf_sect_t *ctfsect, const ctf_sect_t *symsect,
 
   libctf_init_debug();
 
-  if ((ctfsect == NULL) || ((symsect != NULL) && (strsect == NULL)))
+  if ((ctfsect == NULL) || ((symsect != NULL) &&
+			    ((strsect == NULL) && syn_strtab == NULL)))
     return (ctf_set_open_errno (errp, EINVAL));
 
   if (symsect != NULL && symsect->cts_entsize != sizeof (Elf32_Sym) &&
@@ -1466,6 +1492,7 @@ ctf_bufopen (const ctf_sect_t *ctfsect, const ctf_sect_t *symsect,
       fp->ctf_str[CTF_STRTAB_1].cts_strs = strsect->cts_data;
       fp->ctf_str[CTF_STRTAB_1].cts_len = strsect->cts_size;
     }
+  fp->ctf_syn_ext_strtab = syn_strtab;
 
   if (foreign_endian &&
       (err = flip_ctf (hp, fp->ctf_buf)) != 0)
@@ -1592,12 +1619,13 @@ ctf_file_close (ctf_file_t *fp)
 
   if (fp->ctf_strtab.cts_name != _CTF_NULLSTR)
     ctf_free ((char *) fp->ctf_strtab.cts_name);
-
   else if (fp->ctf_data_mmapped)
     ctf_munmap (fp->ctf_data_mmapped, fp->ctf_data_mmapped_len);
 
   ctf_free (fp->ctf_dynbase);
 
+  ctf_dynhash_destroy (fp->ctf_syn_ext_strtab);
+
   ctf_free (fp->ctf_sxlate);
   ctf_free (fp->ctf_txlate);
   ctf_free (fp->ctf_ptrtab);
diff --git a/libctf/ctf-string.c b/libctf/ctf-string.c
index 27bd7c2..44cd447 100644
--- a/libctf/ctf-string.c
+++ b/libctf/ctf-string.c
@@ -20,13 +20,24 @@
 #include <ctf-impl.h>
 #include <string.h>
 
-/* Convert an encoded CTF string name into a pointer to a C string by looking
-  up the appropriate string table buffer and then adding the offset.  */
+/* Convert an encoded CTF string name into a pointer to a C string, using an
+  explicit internal strtab rather than the fp-based one.  */
 const char *
-ctf_strraw (ctf_file_t *fp, uint32_t name)
+ctf_strraw_explicit (ctf_file_t *fp, uint32_t name, ctf_strs_t *strtab)
 {
   ctf_strs_t *ctsp = &fp->ctf_str[CTF_NAME_STID (name)];
 
+  if ((CTF_NAME_STID (name) == CTF_STRTAB_0) && (strtab != NULL))
+    ctsp = strtab;
+
+  /* If this name is in the external strtab, and there is a synthetic strtab,
+     use it in preference.  */
+
+  if (CTF_NAME_STID (name) == CTF_STRTAB_1
+      && fp->ctf_syn_ext_strtab != NULL)
+    return ctf_dynhash_lookup (fp->ctf_syn_ext_strtab,
+			       (void *) (uintptr_t) name);
+
   if (ctsp->cts_strs != NULL && CTF_NAME_OFFSET (name) < ctsp->cts_len)
     return (ctsp->cts_strs + CTF_NAME_OFFSET (name));
 
@@ -34,6 +45,14 @@ ctf_strraw (ctf_file_t *fp, uint32_t name)
   return NULL;
 }
 
+/* Convert an encoded CTF string name into a pointer to a C string by looking
+  up the appropriate string table buffer and then adding the offset.  */
+const char *
+ctf_strraw (ctf_file_t *fp, uint32_t name)
+{
+  return ctf_strraw_explicit (fp, name, NULL);
+}
+
 /* Return a guaranteed-non-NULL pointer to the string with the given CTF
    name.  */
 const char *
@@ -88,11 +107,11 @@ ctf_str_free_atoms (ctf_file_t *fp)
   ctf_dynhash_destroy (fp->ctf_str_atoms);
 }
 
-/* Add a string to the atoms table and return it, or return an existing string
-   if present, copying the passed-in string.  Returns NULL only when out of
-   memory (and do not touch the passed-in string in that case).  Possibly
-   augment the ref list with the passed-in ref.  */
-static const char *
+/* Add a string to the atoms table, copying the passed-in string.  Return the
+   atom added. Return NULL only when out of memory (and do not touch the
+   passed-in string in that case).  Possibly augment the ref list with the
+   passed-in ref.  */
+static ctf_str_atom_t *
 ctf_str_add_ref_internal (ctf_file_t *fp, const char *str,
 			  int add_ref, uint32_t *ref)
 {
@@ -116,7 +135,7 @@ ctf_str_add_ref_internal (ctf_file_t *fp, const char *str,
 	  ctf_list_append (&atom->csa_refs, aref);
 	  fp->ctf_str_num_refs++;
 	}
-      return atom->csa_str;
+      return atom;
     }
 
   if ((atom = ctf_alloc (sizeof (struct ctf_str_atom))) == NULL)
@@ -136,7 +155,7 @@ ctf_str_add_ref_internal (ctf_file_t *fp, const char *str,
       ctf_list_append (&atom->csa_refs, aref);
       fp->ctf_str_num_refs++;
     }
-  return newstr;
+  return atom;
 
  oom:
   ctf_free (atom);
@@ -150,9 +169,48 @@ ctf_str_add_ref_internal (ctf_file_t *fp, const char *str,
 const char *
 ctf_str_add (ctf_file_t *fp, const char *str)
 {
-  if (str)
-    return ctf_str_add_ref_internal (fp, str, FALSE, 0);
-  return NULL;
+  ctf_str_atom_t *atom;
+  if (!str)
+    return NULL;
+
+  atom = ctf_str_add_ref_internal (fp, str, FALSE, 0);
+  if (!atom)
+    return NULL;
+
+  return atom->csa_str;
+}
+
+/* Like ctf_str_add(), but additionally augment the atom's refs list with the
+   passed-in ref, whether or not the string is already present.  There is no
+   attempt to deduplicate the refs list (but duplicates are harmless).  */
+const char *
+ctf_str_add_ref (ctf_file_t *fp, const char *str, uint32_t *ref)
+{
+  ctf_str_atom_t *atom;
+  if (!str)
+    return NULL;
+
+  atom = ctf_str_add_ref_internal (fp, str, TRUE, ref);
+  if (!atom)
+    return NULL;
+
+  return atom->csa_str;
+}
+
+/* Add an external strtab reference at OFFSET.  */
+const char *
+ctf_str_add_external (ctf_file_t *fp, const char *str, uint32_t offset)
+{
+  ctf_str_atom_t *atom;
+  if (!str)
+    return NULL;
+
+  atom = ctf_str_add_ref_internal (fp, str, FALSE, 0);
+  if (!atom)
+    return NULL;
+
+  atom->csa_offset = CTF_SET_STID (offset, CTF_STRTAB_1);
+  return atom->csa_str;
 }
 
 /* A ctf_dynhash_iter_remove() callback that removes atoms later than a given
@@ -173,17 +231,6 @@ ctf_str_rollback (ctf_file_t *fp, ctf_snapshot_id_t id)
   ctf_dynhash_iter_remove (fp->ctf_str_atoms, ctf_str_rollback_atom, &id);
 }
 
-/* Like ctf_str_add(), but additionally augment the atom's refs list with the
-   passed-in ref, whether or not the string is already present.  There is no
-   attempt to deduplicate the refs list (but duplicates are harmless).  */
-const char *
-ctf_str_add_ref (ctf_file_t *fp, const char *str, uint32_t *ref)
-{
-  if (str)
-    return ctf_str_add_ref_internal (fp, str, TRUE, ref);
-  return NULL;
-}
-
 /* An adaptor around ctf_purge_atom_refs.  */
 static void
 ctf_str_purge_one_atom_refs (void *key _libctf_unused_, void *value,
@@ -238,7 +285,11 @@ ctf_str_count_strtab (void *key _libctf_unused_, void *value,
   ctf_str_atom_t *atom = (ctf_str_atom_t *) value;
   ctf_strtab_write_state_t *s = (ctf_strtab_write_state_t *) arg;
 
-  s->strtab->cts_len += strlen (atom->csa_str) + 1;
+  /* We only factor in the length of items that have no offset:
+     other items are in the external strtab.  They still contribute to the
+     total count, though, because we still have to sort them.  */
+  if (!atom->csa_offset)
+    s->strtab->cts_len += strlen (atom->csa_str) + 1;
   s->strtab_count++;
 }
 
@@ -268,8 +319,10 @@ ctf_str_sort_strtab (const void *a, const void *b)
 }
 
 /* Write out and return a strtab containing all strings with recorded refs,
-   adjusting the refs to refer to the corresponding string.  The returned
-   strtab may be NULL on error.  */
+   adjusting the refs to refer to the corresponding string.  The returned strtab
+   may be NULL on error.  Also populate the synthetic strtab with mappings from
+   external strtab offsets to names, so we can look them up with ctf_strptr().
+   Only external strtab offsets with references are added.  */
 ctf_strs_writable_t
 ctf_str_write_strtab (ctf_file_t *fp)
 {
@@ -279,6 +332,7 @@ ctf_str_write_strtab (ctf_file_t *fp)
   ctf_strtab_write_state_t s;
   ctf_str_atom_t **sorttab;
   size_t i;
+  int any_external = 0;
 
   memset (&strtab, 0, sizeof (struct ctf_strs_writable));
   memset (&s, 0, sizeof (struct ctf_strtab_write_state));
@@ -300,7 +354,7 @@ ctf_str_write_strtab (ctf_file_t *fp)
   /* Sort the strtab.  Force the null string to be first.  */
   sorttab = calloc (s.strtab_count, sizeof (ctf_str_atom_t *));
   if (!sorttab)
-      return strtab;
+    goto oom;
 
   sorttab[0] = nullstr;
   s.i = 1;
@@ -312,19 +366,58 @@ ctf_str_write_strtab (ctf_file_t *fp)
 	 ctf_str_sort_strtab);
 
   if ((strtab.cts_strs = ctf_alloc (strtab.cts_len)) == NULL)
-    {
-      free (sorttab);
-      return strtab;
-    }
+    goto oom_sorttab;
+
+  if (!fp->ctf_syn_ext_strtab)
+    fp->ctf_syn_ext_strtab = ctf_dynhash_create (ctf_hash_integer,
+						 ctf_hash_eq_integer,
+						 NULL, NULL);
+  if (!fp->ctf_syn_ext_strtab)
+    goto oom_strtab;
 
-  /* Update the strtab, and all refs.  */
+  /* Update all refs: also update the strtab appropriately.  */
   for (i = 0; i < s.strtab_count; i++)
     {
-      strcpy (&strtab.cts_strs[cur_stroff], sorttab[i]->csa_str);
-      ctf_str_update_refs (sorttab[i], cur_stroff);
-      cur_stroff += strlen (sorttab[i]->csa_str) + 1;
+      if (sorttab[i]->csa_offset)
+	{
+	  /* External strtab entry: populate the synthetic external strtab.
+
+	     This is safe because you cannot ctf_rollback to before the point
+	     when a ctf_update is done, and the strtab is written at ctf_update
+	     time.  So any atoms we reference here are sure to stick around
+	     until ctf_file_close.  */
+
+	  any_external = 1;
+	  ctf_str_update_refs (sorttab[i], sorttab[i]->csa_offset);
+	  if (ctf_dynhash_insert (fp->ctf_syn_ext_strtab,
+				  (void *) (uintptr_t) sorttab[i]->csa_offset,
+				  (void *) sorttab[i]->csa_str) < 0)
+	    goto oom_strtab;
+	}
+      else
+	{
+	  /* Internal strtab entry: actually add to the string table.  */
+
+	  ctf_str_update_refs (sorttab[i], cur_stroff);
+	  strcpy (&strtab.cts_strs[cur_stroff], sorttab[i]->csa_str);
+	  cur_stroff += strlen (sorttab[i]->csa_str) + 1;
+	}
     }
   free (sorttab);
 
+  if (!any_external)
+    {
+      ctf_dynhash_destroy (fp->ctf_syn_ext_strtab);
+      fp->ctf_syn_ext_strtab = NULL;
+    }
+
+  return strtab;
+
+ oom_strtab:
+  free (strtab.cts_strs);
+  strtab.cts_strs = NULL;
+ oom_sorttab:
+  free (sorttab);
+ oom:
   return strtab;
 }