Switch from semaphores to message queue to share workload. (#305)
Using semaphores for work-sharing isn't ideal and brings some complexity and inefficiencies (each worker process loops over the whole table array). Now that we have introduced Sys V queues, use that mechanism instead to share the workload between table-data COPY workers. We still need to use a semaphore to create the lockfile and donefile so that we can track progress, resume from interrupted operations, and discover if all the partitions of a single table have been processed already. That said, we don't have to deal with same-table concurrency using the semaphore and files, the queue solves that for us.
parent
f060895b
Please register or sign in to comment