libctf: allow the header to change between versions

libctf supports dynamic upgrading of the type table as file format
versions change, but before now has not supported changes to the CTF
header.  Doing this is complicated by the baroque storage method used:
the CTF header is kept prepended to the rest of the CTF data, just as
when read from the file, and written out from there, and is
endian-flipped in place.

This makes accessing it needlessly hard and makes it almost impossible
to make the header larger if we add fields.  The general storage
machinery around the malloced ctf pointer (the 'ctf_base') is also
overcomplicated: the pointer is sometimes malloced locally and sometimes
assigned from a parameter, so freeing it requires checking to see if
that parameter was used, needlessly coupling ctf_bufopen and
ctf_file_close together.

So split the header out into a new ctf_file_t.ctf_header, which is
written out explicitly: squeeze it out of the CTF buffer whenever we
reallocate it, and use ctf_file_t.ctf_buf to skip past the header when
we do not need to reallocate (when no upgrading or endian-flipping is
required).  We now track whether the CTF base can be freed explicitly
via a new ctf_dynbase pointer which is non-NULL only when freeing is
possible.

With all this done, we can upgrade the header on the fly and add new
fields as desired, via a new upgrade_header function in ctf-open.
As with other forms of upgrading, libctf upgrades older headers
automatically to the latest supported version at open time.

For a first use of this field, we add a new string field cth_cuname, and
a corresponding setter/getter pair ctf_cuname_set and ctf_cuname: this
is used by debuggers to determine whether a CTF section's types relate
to a single compilation unit, or to all compilation units in the
program.  (Types with ambiguous definitions in different CUs have only
one of these types placed in the top-level shared .ctf container: the
rest are placed in much smaller per-CU containers, which have the shared
container as their parent.  Since CTF must be useful in the absence of
DWARF, we store the names of the relevant CUs ourselves, so the debugger
can look them up.)

v5: fix tabdamage.

include/
	* ctf-api.h (ctf_cuname): New function.
	(ctf_cuname_set): Likewise.
	* ctf.h: Improve comment around upgrading, no longer
	implying that v2 is the target of upgrades (it is v3 now).
	(ctf_header_v2_t): New, old-format header for backward
	compatibility.
	(ctf_header_t): Add cth_cuname: this is the first of several
	header changes in format v3.
libctf/
	* ctf-impl.h (ctf_file_t): New fields ctf_header, ctf_dynbase,
	ctf_cuname, ctf_dyncuname: ctf_base and ctf_buf are no longer const.
	* ctf-open.c (ctf_set_base): Preserve the gap between ctf_buf and
	ctf_base: do not assume that it is always sizeof (ctf_header_t).
	Print out ctf_cuname: only print out ctf_parname if set.
	(ctf_free_base): Removed, ctf_base is no longer freed: free
	ctf_dynbase instead.
	(ctf_set_version): Fix spacing.
	(upgrade_header): New, in-place header upgrading.
	(upgrade_types): Rename to...
	(upgrade_types_v1): ... this.  Free ctf_dynbase, not ctf_base.  No
	longer track old and new headers separately.  No longer allow for
	header sizes explicitly: squeeze the headers out on upgrade (they
	are preserved in fp->ctf_header).  Set ctf_dynbase, ctf_base and
	ctf_buf explicitly.  Use ctf_free, not ctf_free_base.
	(upgrade_types): New, also handle ctf_parmax updating.
	(flip_header): Flip ctf_cuname.
	(flip_types): Flip BUF explicitly rather than deriving BUF from
	BASE.
	(ctf_bufopen): Store the header in fp->ctf_header.  Correct minimum
	required alignment of objtoff and funcoff.  No longer store it in
	the ctf_buf unless that buf is derived unmodified from the input.
	Set ctf_dynbase where ctf_base is dynamically allocated. Drop locals
	that duplicate fields in ctf_file: move allocation of ctf_file
	further up instead.  Call upgrade_header as needed.  Move
	version-specific ctf_parmax initialization into upgrade_types.  More
	concise error handling.
	(ctf_file_close): No longer test for null pointers before freeing.
	Free ctf_dyncuname, ctf_dynbase, and ctf_header.  Do not call
	ctf_free_base.
	(ctf_cuname): New.
	(ctf_cuname_set): New.
	* ctf-create.c (ctf_update): Populate ctf_cuname.
	(ctf_gzwrite): Write out the header explicitly.  Remove obsolescent
	comment.
	(ctf_write): Likewise.
	(ctf_compress_write): Get the header from ctf_header, not ctf_base.
	Fix the compression length: fp->ctf_size never counted the CTF
	header.  Simplify the compress call accordingly.

3 files changed, 36 insertions, 7 deletions

diff --git a/include/ChangeLog b/include/ChangeLog
index 41adb65..de93394 100644
--- a/include/ChangeLog
+++ b/include/ChangeLog
@@ -1,3 +1,14 @@
+2019-07-06  Nick Alcock  <nick.alcock@oracle.com>
+
+	* ctf-api.h (ctf_cuname): New function.
+	(ctf_cuname_set): Likewise.
+	* ctf.h: Improve comment around upgrading, no longer
+	implying that v2 is the target of upgrades (it is v3 now).
+	(ctf_header_v2_t): New, old-format header for backward
+	compatibility.
+	(ctf_header_t): Add cth_cuname: this is the first of several
+	header changes in format v3.
+
 2019-09-23  Alan Modra  <amodra@gmail.com>
 
 	* bfdlink.h (struct bfd_section_already_linked): Forward declare.
diff --git a/include/ctf-api.h b/include/ctf-api.h
index 1a0d47e..28256a3 100644
--- a/include/ctf-api.h
+++ b/include/ctf-api.h
@@ -258,6 +258,8 @@ extern void ctf_file_close (ctf_file_t *);
 extern int ctf_arc_write (const char *, ctf_file_t **, size_t,
 			  const char **, size_t);
 
+extern const char *ctf_cuname (ctf_file_t *);
+extern void ctf_cuname_set (ctf_file_t *, const char *);
 extern ctf_file_t *ctf_parent_file (ctf_file_t *);
 extern const char *ctf_parent_name (ctf_file_t *);
 extern void ctf_parent_name_set (ctf_file_t *, const char *);
diff --git a/include/ctf.h b/include/ctf.h
index 2b35781..7e00005 100644
--- a/include/ctf.h
+++ b/include/ctf.h
@@ -126,11 +126,26 @@ typedef struct ctf_preamble
   unsigned char ctp_flags;	/* Flags (see below).  */
 } ctf_preamble_t;
 
+typedef struct ctf_header_v2
+{
+  ctf_preamble_t cth_preamble;
+  uint32_t cth_parlabel;	/* Ref to name of parent lbl uniq'd against.  */
+  uint32_t cth_parname;		/* Ref to basename of parent.  */
+  uint32_t cth_lbloff;		/* Offset of label section.  */
+  uint32_t cth_objtoff;		/* Offset of object section.  */
+  uint32_t cth_funcoff;		/* Offset of function section.  */
+  uint32_t cth_varoff;		/* Offset of variable section.  */
+  uint32_t cth_typeoff;		/* Offset of type section.  */
+  uint32_t cth_stroff;		/* Offset of string section.  */
+  uint32_t cth_strlen;		/* Length of string section in bytes.  */
+} ctf_header_v2_t;
+
 typedef struct ctf_header
 {
   ctf_preamble_t cth_preamble;
   uint32_t cth_parlabel;	/* Ref to name of parent lbl uniq'd against.  */
   uint32_t cth_parname;		/* Ref to basename of parent.  */
+  uint32_t cth_cuname;		/* Ref to CU name (may be 0).  */
   uint32_t cth_lbloff;		/* Offset of label section.  */
   uint32_t cth_objtoff;		/* Offset of object section.  */
   uint32_t cth_funcoff;		/* Offset of function section.  */
@@ -148,13 +163,14 @@ typedef struct ctf_header
 
 /* Data format version number.  */
 
-/* v1 upgraded to v2 is not quite the same as native v2 (the boundary between
-   parent and child types is different), and you can write it out again via
-   ctf_compress_write(), so we must track whether the thing was originally v1 or
-   not.  If we were writing the header from scratch, we would add a *pair* of
-   version number fields to allow for this, but this will do for now.  (A flag
-   will not do, because we need to encode both the version we came from and the
-   version we went to, not just "we were upgraded".) */
+/* v1 upgraded to a later version is not quite the same as the native form,
+   because the boundary between parent and child types is different but not
+   recorded anywhere, and you can write it out again via ctf_compress_write(),
+   so we must track whether the thing was originally v1 or not.  If we were
+   writing the header from scratch, we would add a *pair* of version number
+   fields to allow for this, but this will do for now.  (A flag will not do,
+   because we need to encode both the version we came from and the version we
+   went to, not just "we were upgraded".) */
 
 # define CTF_VERSION_1 1
 # define CTF_VERSION_1_UPGRADED_3 2