Avoid cache race in bfd_check_format_matches

Running the gdb test suite with the thread sanitizer enabled shows a
race when bfd_check_format_matches and bfd_cache_close_all are called
simultaneously on different threads.

This patch fixes this race by having bfd_check_format_matches
temporarily remove the BFD from the file descriptor cache -- leaving
it open while format-checking proceeds.

In this setup, the BFD client is responsible for closing the BFD again
on the "checking" thread, should that be desired.  gdb does this by
calling bfd_cache_close in the relevant worker thread.

An earlier version of this patch omitted the "possibly_cached" helper
function.  However, this ran into crashes in the binutils test suite
involving the archive-checking abort in bfd_cache_lookup_worker.  I do
not understand the purpose of this check, so I've simply had the new
function work around it.  I couldn't find any comments explaining this
situation, either.  I suspect that there may still be races related to
this case, but I don't think I have access to the platforms where gdb
deals with archives.

Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=31264

1 files changed, 6 insertions, 0 deletions

diff --git a/bfd/bfd.c b/bfd/bfd.c
index ace2f67..ae79c64 100644
--- a/bfd/bfd.c
+++ b/bfd/bfd.c
@@ -307,6 +307,12 @@ CODE_FRAGMENT
 .  {* LTO object type.  *}
 .  ENUM_BITFIELD (bfd_lto_object_type) lto_type : 2;
 .
+.  {* Set if this BFD is currently being processed by
+.     bfd_check_format_matches.  This is checked by the cache to
+.     avoid closing the BFD in this case.  This should only be
+.     examined or modified while the BFD lock is held.  *}
+.  unsigned int in_format_matches : 1;
+.
 .  {* Set to dummy BFD created when claimed by a compiler plug-in
 .     library.  *}
 .  bfd *plugin_dummy_bfd;