Commit 8cf9d785 authored by Matthew Fernandez's avatar Matthew Fernandez
Browse files

fix final thread seeing a stale local_seen

There was a race condition involving the rendezvous opt out and set expansion. I
can think of two ways in which the same problem occurs. Firstly:

  1. Thread A calls error() and begins the exit flow
  2. Other threads begin a set expansion and migration
  3. Thread A opts out of the rendezvous protocol
  4. Other threads finish the migration and pass through the rendezvous point

Secondly:

  1. Thread A calls error() and begins the exit flow
  2. Other threads begin a set expansion and migration
  3. Other threads finish the migration and arrive at the rendezvous point
  4. Thread A opts out of the rendezvous protocol

Observe that in both these scenarios the reference count of the old seen set
never drops to zero because the rendezvous opt out does not decrement it. This
means that no thread thinks it is the leader and the global_seen pointer doesn't
get updated. As a result, thread A now has a stale local_seen pointer that
points to a set full of tombstones.

We fix this with three relevant changes in this commit:

  1. Shift the next seen set into global_seen in the rendezvous interstice
     instead of prior. This ensures that this action is always performed by a
     thread when we pass through the rendezvous point.
  2. Re-acquire a local_seen pointer in exit_with() after thread joining. This
     ensures that the final thread has a local_seen that points at the currently
     active global_seen set.
  3. Drop the reference count to the global_seen set when opting out of the
     rendezvous protocol. This is not strictly necessary, but it allows us to
     preserve the property (and assertion) in refcounted_ptr_shift that the
     reference count is zero at this point.

The problem addressed here was never observed in the wild but rather something I
realised could happen by thinking about the concurrency interactions. The
changes described above should have no measurable impact on runtime or memory
usage.

Github: closes #124 "can the final thread see a stale local_seen?"
parent 335e8253
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment