Review logical decoding client tracking of LSNs. (#502)
* Review logical decoding client tracking of LSNs. When replaying from file, ensure that the last transaction of a file is commited with synchronous_commit on, allowing better tracking of replication progress. Also, improve our way to keep track of the previousLSN and log when we reach endpos (KEEPALIVE, COMMIT, ENDPOS) to help get a sense of what's happening. By setting previousLSN to the proper value even when reaching endpos at a KEEPALIVE message that we skip, we allow the overall system to exit properly in case when there is no activity on the source server. * Enhance clean_results() to avoid blocking calls to PQgetResult(). In a debug session with gdb, now enabled in our docker-compose setup, it was apparent that a process would block on a PQgetResult() call done in the context of clear_results() when an error would happen during the logical replication streaming. Fix both the clear_results() behavior and also refrain from calling it on the error path, just finish the connection without trying to get possibly pending results that we're not going to need anyway. * Review process startup init and setting of endpos / --endpos. When the command line option is used, we should not take the value that is fetched from the source pgcopydb.sentinel table. Also protect some processes against having already reached endpos at startup time. In passing, try to test for Broken PIPE upon connection retries. Testing is not conclusive on that parts, but it seems harmless. * Properly reset stdIn and stdOut booleans to false when needed. Now that we have pro-active actions implemented to check if the PIPE might be broken we need to pay attention to these variables. Also in passing have the catchup process between follow loops happen in sub-processes too. That way the process model stays the same even in that area of the code: the actual processing is done by sub-processes. * Review Dockerfiles: replace psutils with psmisc. psutils: PostScript document handling utilities psmisc: utilities that use the proc file system (including pstree) also add htop which is useful at displaying a process tree
parent
13470c4f
Please register or sign in to comment