libctf, ld: properly deduplicate function types

Some type kinds in CTF (functions, arrays, pointers, slices, and
cvr-quals) are intrinsically nameless: the ctt_name field in the CTF
is always zero, and the libctf API provides no way to set a name.
But the compiler can and does sometimes set names for some of these
kinds: in particular, the name it sets on CTF_K_FUNCTION types is the
means it uses to force the name of the function into the string table
so that it can point at it from the function info section.

So null out the name at hashing time so that the deduplicator can
correctly detect that e.g. function types identical but for name should
be considered truly identical, since they will not have a name when the
deduplicator re-emits them into the output.

ld/ChangeLog
2020-11-20  Nick Alcock  <nick.alcock@oracle.com>

	* testsuite/ld-ctf/data-func-conflicted.d: Shrink the expected
	size of the type section now that function types are being
	deduplicated properly.

libctf/ChangeLog
2020-11-20  Nick Alcock  <nick.alcock@oracle.com>

	* ctf-dedup.c (ctf_dedup_rhash_type): Null out the names of nameless
	type kinds, just in case the input has named them.

4 files changed, 33 insertions, 6 deletions

diff --git a/ld/ChangeLog b/ld/ChangeLog
index 8b088b4..5e81fa2 100644
--- a/ld/ChangeLog
+++ b/ld/ChangeLog
@@ -1,5 +1,11 @@
 2020-11-20  Nick Alcock  <nick.alcock@oracle.com>
 
+	* testsuite/ld-ctf/data-func-conflicted.d: Shrink the expected
+	size of the type section now that function types are being
+	deduplicated properly.
+
+2020-11-20  Nick Alcock  <nick.alcock@oracle.com>
+
 	* testsuite/ld-ctf/array.d: Adjust for nonzero flags word and
 	public symbols in the data section rather than variables: use
 	sysv hash style to keep test results the same on non-GNU targets.
diff --git a/ld/testsuite/ld-ctf/data-func-conflicted.d b/ld/testsuite/ld-ctf/data-func-conflicted.d
index 626bfef..1fa8bb2 100644
--- a/ld/testsuite/ld-ctf/data-func-conflicted.d
+++ b/ld/testsuite/ld-ctf/data-func-conflicted.d
@@ -16,7 +16,7 @@ Contents of CTF section \.ctf:
     Data object section:	.* \(0xc bytes\)
     Function info section:	.* \(0x40 bytes\)
     Object index section:	.* \(0xc bytes\)
-    Type section:	.* \(0x228 bytes\)
+    Type section:	.* \(0xe8 bytes\)
     String section:	.*
 #...
   Data objects:
diff --git a/libctf/ChangeLog b/libctf/ChangeLog
index e81db2c..e571404 100644
--- a/libctf/ChangeLog
+++ b/libctf/ChangeLog
@@ -1,5 +1,10 @@
 2020-11-20  Nick Alcock  <nick.alcock@oracle.com>
 
+	* ctf-dedup.c (ctf_dedup_rhash_type): Null out the names of nameless
+	type kinds, just in case the input has named them.
+
+2020-11-20  Nick Alcock  <nick.alcock@oracle.com>
+
 	* ctf-dump.c (ctf_dump_header): Dump the new flags bits and the index
 	section lengths.
 	(ctf_dump_objts): Report indexed sections.  Also dump functions.  Use
diff --git a/libctf/ctf-dedup.c b/libctf/ctf-dedup.c
index 3e95af0..b0be5a7 100644
--- a/libctf/ctf-dedup.c
+++ b/libctf/ctf-dedup.c
@@ -638,6 +638,27 @@ ctf_dedup_rhash_type (ctf_dict_t *fp, ctf_dict_t *input, ctf_dict_t **inputs,
 	&& ctf_dedup_record_origin (fp, input_num, decorated, type_id) < 0)
       return NULL;				/* errno is set for us.  */
 
+#ifdef ENABLE_LIBCTF_HASH_DEBUGGING
+  ctf_dprintf ("%lu: hashing thing with ID %i/%lx (kind %i): %s.\n",
+	       depth, input_num, type, kind, name ? name : "");
+#endif
+
+  /* Some type kinds don't have names: the API provides no way to set the name,
+     so the type the deduplicator outputs will be nameless even if the input
+     somehow has a name, and the name should not be mixed into the hash.  */
+
+  switch (kind)
+    {
+    case CTF_K_POINTER:
+    case CTF_K_ARRAY:
+    case CTF_K_FUNCTION:
+    case CTF_K_VOLATILE:
+    case CTF_K_CONST:
+    case CTF_K_RESTRICT:
+    case CTF_K_SLICE:
+      name = NULL;
+    }
+
   /* Mix in invariant stuff, transforming the type kind if needed.  Note that
      the vlen is *not* hashed in: the actual variable-length info is hashed in
      instead, piecewise.  The vlen is not part of the type, only the
@@ -647,11 +668,6 @@ ctf_dedup_rhash_type (ctf_dict_t *fp, ctf_dict_t *input, ctf_dict_t **inputs,
      *other types in the same TU* with the same name: so two types can easily
      have distinct nonroot flags, yet be exactly the same type.*/
 
-#ifdef ENABLE_LIBCTF_HASH_DEBUGGING
-  ctf_dprintf ("%lu: hashing thing with ID %i/%lx (kind %i): %s.\n",
-	       depth, input_num, type, kind, name ? name : "");
-#endif
-
   ctf_sha1_init (&hash);
   if (name)
     ctf_dedup_sha1_add (&hash, name, strlen (name) + 1, "name", depth);