Implement a sentinel. (#69)
* Implement a sentinel. The sentinel is a table on the source database with 3 columns to share the LSN start position for the logical decoding receiver, the LSN end position where to stop receiving and applying logical decoding changes, and finally the apply boolean which can be set to true when the catching-up process is allowed to start processing pre-fetched events. The idea is to set the sentinel apply boolean as soon as the base copy finishes, and to allow pgcopydb users and test suite to set and endpos after the start of the follow processes. * Assorted fixes. - review the test files to use the new commands facilities - only use the sentinel in the catchup command, not the apply command - filter out CDC for pgcopydb.* tables (sentinel) - allow retrying until the streaming context files have been created * Use the pgcopydb sentinel table startpos and endpos in follow command. Rather than manually fetching the current LSN on the source database and then passing that as an argument to pgcopydb follow --endpos, we can now use the pgcopydb stream sentinel set endpos --current command. * Use the pgcopydb sentinel to track progress, and sync as we go. We need to sync with the sentinel changes in case an endpos has been set concurrently to the pgcopydb follow processing. We can also use the sentinel shared table as a way to exchange information between the replay process and the receive process, such as the replay_lsn. * Publish replay_lsn feedback to the replication stream. * Implement support for early WAL file switching, improve tests. Now the tests are running a separate instance for the injection of SQL commands, and the sentinel endpos is driven from outside the pgcopydb follow process and command. * Review and adjust documentation for pgcopydb follow etc. This PR introduces a bunch of new commands and also changes in how some commands are made available. Finally edit the docs to reflect the new CLI.
parent
6a6530ea
Please register or sign in to comment