Unverified Commit 974972e8 authored by Dimitri Fontaine's avatar Dimitri Fontaine Committed by GitHub
Browse files

Implement a sentinel. (#69)

* Implement a sentinel.

The sentinel is a table on the source database with 3 columns to share the
LSN start position for the logical decoding receiver, the LSN end position
where to stop receiving and applying logical decoding changes, and finally
the apply boolean which can be set to true when the catching-up process is
allowed to start processing pre-fetched events.

The idea is to set the sentinel apply boolean as soon as the base copy
finishes, and to allow pgcopydb users and test suite to set and endpos after
the start of the follow processes.

* Assorted fixes.

  - review the test files to use the new commands facilities
  - only use the sentinel in the catchup command, not the apply command
  - filter out CDC for pgcopydb.* tables (sentinel)
  - allow retrying until the streaming context files have been created

* Use the pgcopydb sentinel table startpos and endpos in follow command.

Rather than manually fetching the current LSN on the source database and
then passing that as an argument to pgcopydb follow --endpos, we can now use
the pgcopydb stream sentinel set endpos --current command.

* Use the pgcopydb sentinel to track progress, and sync as we go.

We need to sync with the sentinel changes in case an endpos has been set
concurrently to the pgcopydb follow processing. We can also use the sentinel
shared table as a way to exchange information between the replay process and
the receive process, such as the replay_lsn.

* Publish replay_lsn feedback to the replication stream.

* Implement support for early WAL file switching, improve tests.

Now the tests are running a separate instance for the injection of SQL
commands, and the sentinel endpos is driven from outside the pgcopydb follow
process and command.

* Review and adjust documentation for pgcopydb follow etc.

This PR introduces a bunch of new commands and also changes in how some
commands are made available. Finally edit the docs to reflect the new CLI.
parent 6a6530ea
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment