Commit b93ad26e authored by Matthew Fernandez's avatar Matthew Fernandez
Browse files

implement thread-safe set expansion

This commit adapts some ideas from the paper Maier et al, "Concurrent Hash
Tables: Fast and General(?)!" in arXiv 2016. The set expansion now happens in
coordination with other threads. Namely:

  * If you insert into the set and you discover it over-threshold, you initiate
    the expansion;
  * If you discover an expansion is already in progress, you join the cohort of
    threads migrating entries from the old set to the new one; and
  * If you attempt an expansion and find one already in progress, you fall back
    to assisting in the migration.

There are a few warts to this commit, which is why it does not completely close
Github #30:

  * Migration does not happen exactly in accordance with Maier et al's
    algorithm. We break the set into 4K chunks and migrate these, without regard
    to the semantic structure of the set. Following more closely to Maier et
    al's design, it would be possible to "weaken" some migration operations from
    atomic operations to normal stores.
  * There's some -mcx16 related mess. See some code comments and
    https://gcc.gnu.org/bugzilla/show_bug.cgi?id=80878 for more on this.
  * We're lacking a line-of-sight to shutdown-during-expansion. To unpack that a
    bit, it's possible for one thread to have exhausted its search for eligible
    expansion states while another thread is expanding the seen set. Under the
    inertia of the current design, this will result in the first thread exiting
    while the second thread becomes the leader of a new expansion involving one
    less thread than it thinks it leads. The end result is that the expansion
    process will deadlock waiting on the thread who has exited. This is not
    impossible to resolve, but will involve some changes.

It should be spelled out that there is still no concurrency. When we refactored
from the C++ checker to the C checker, multithreaded execution was dropped. The
present changes are preparation for once again going multithreaded. This commit
should be the last thing we need to make the seen set thread-safe. The next
hurdle is load balancing between thread-local pending state queues.

Related to Github #30 "TODO: thread-safe set expansion"
parent ab2325b4
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment