This extension requires specific pre and post pg_restore steps to be
implemented, and also seems to require pgcopydb to use a superuser role to
be able to fetch the data and use pg_dump.

- When the filtering results in an empty list of selected tables, make
    sure we don't enter infinite loop waiting for events that won't happen.

  - Make sure to prepare the pgcopydb_table_size (temporary) table when
    doing the pgcopydb copy ... commands that need it.

  - Ensure proper memory allocation and calloc() error detection when
    printing the summary tables.

  - Work around what looks like a compiler bug when int errors = 0; actually
    sets the errors counter to 32759 on macOs. For that the COPY workers are
	now always in "fail-fast" mode, we can't count the errors anymore.

* Fix file rotation issue during streaming.

Fixes a bug that occurs when multiple transactions have BEGIN and other
statements in one WAL file, while their COMMITs are in the next file. The
issue arises when pgcopydb receives the COMMIT of the first transaction,
causing it to move to the next file. However, upon receiving the BEGIN of
the next transaction, it incorrectly goes back to the previous file,
resulting in an inconsistent file rotation pattern.

To address this issue, this commit introduces tracking the LSN of the last
message written to the file. With this, file rotation is skipped if the
incoming message has an LSN less than the lastWrittenLSN, ensuring that
writes consistently occur in the current or next file.

* Rename lastWrite to lastWriteTime.

* Fix missing commit_lsn for continued transaction.

When a transaction is split into multiple JSON WAL files, the transformed
BEGIN statement does not include the commit_lsn since the COMMIT statement
appears in the next file. This poses a challenge during the apply process,
as the commit_lsn is necessary to determine when to start applying the
current and subsequent changes.

To resolve this issue, this fix parse the commit_lsn during the apply phase
from a transaction metadata file. This file is created during the transform
phase as soon as the COMMIT message is processed. However, this also
implies that the apply process must wait for the creation of this file
before it can begin applying the changes.

Fixes: #299

This ensures that Postgres object names are properly handled even when they
may contain multiple lines, which is accepted, possible... and then found in
the field.

In passing, ensure that we VACUUM ANALYZE tables without indexes. It's
almost relevant to that patch because the failure mode of multiple-lines
table names would break our vacuum module.

Using semaphores for work-sharing isn't ideal and brings some complexity and
inefficiencies (each worker process loops over the whole table array). Now
that we have introduced Sys V queues, use that mechanism instead to share
the workload between table-data COPY workers.

We still need to use a semaphore to create the lockfile and donefile so that
we can track progress, resume from interrupted operations, and discover if
all the partitions of a single table have been processed already. That said,
we don't have to deal with same-table concurrency using the semaphore and
files, the queue solves that for us.

This is currently a moving target. Version 3.3.3 adds topology_id_seq as an
Extension Configuration Table, and pgcopydb is not ready to handle a
sequence there.

This fixes the issues where we skip applying I/U/D/T/C leading to the
warning "there is already a transaction in progress" when a txn continues
to next file and first statement of that file is not a BEGIN or KEEPALIVE
statement. In that case, `reachedStartPos` is never set to true and apply
never starts.

This happens because `reachedStartPos` is set to `false` everytime a new
file is processed.

During prefetching, when a transaction continues to the next file and the
first line of the next file (which is also the only remaining statement of
the transaction) is a COMMIT message, it is incorrectly interpreted as an
empty transaction. This fix addresses that issue.

This fixes the missing `xid` and `timestamp` for continued transactions,
```sql
COMMIT; -- {"xid":0,"lsn":"6/A50001C8","timestamp":""}
```

This timestamp is used in `pgsql_replication_origin_xact_setup` which we
update along with `COMMIT;` statement. So it makes more sense to put
COMMIT's `timestamp` as origin timestamp.

Retaining the current update with `BEGIN` action as well. This avoids
`timestamp` being empty for `BEGIN` SQL statement in the file when COMMIT
spills to next SQL file.

Check the number of messages in the queue before entering a message receive
loop that won't exit before reading in the STOP message. When the queue is
empty, this turns into an infinite loop.

See #275 

This requires loading more of the Postgres catalog information in-memory.
That's the cost to fix extension configuration tables that have changed
schema between source and target, and to later support schema changes at
pgcopydb time.

When the previous streaming mode was REPLAY and we attempt to empty the
transform queue using `stream_transform_from_queue`, the function waits
indefinitely for `QMSG_TYPE_STOP` (or any message) because the queue has
already been emptied.

It turns out that we can't just call pg_create_logical_replication_slot() in
a transaction that uses a snapshot to sync the initial copy of the data with
the change data capture. We need to use the CREATE_REPLICATION_SLOT logical
replication command with the RESERVE_WAL true SNAPSHOT 'export' options.

* Add a new test case that fails because of missing data around replication slot creation.

* Implement tracking of --plugin, --origin, and --slot-name options.

We already tracked --snapshot so that we can --resume and continue operating
with the same snapshot as before, and we should really have the same
facility for the rest of the replication based options.

* In --follow mode, export the snapshot via CREATE_REPLICATION_SLOT.

The previous code would set transaction snapshot to a previously exported
snapshot and then create the logical replication slot, but that creates a
replication gap. Postgres supports only one way to avoid that gap, which
consists of exporting the snapshot used to create the replication slot.

In passing, register the full information we get from creating the
replication slot to file, in such a way that we can re-use that file and
information later, including the consistent_point LSN that is associated
with the replication slot... and the snapshot.

As a result, the following commands have been removed from the CLI:

    pgcopydb stream create slot
    pgcopydb stream create origin
    pgcopydb stream drop slot
    pgcopydb stream drop origin

The only way to create a replication slot with pgcopydb is now the command
`pgcopydb snapshot --follow`, and the replication origin on the target must
be created with the consistent_point LSN returned by the same command.

* Fix unit tests to adjust to the new API/CLI.

The creation of a replication slot is not done in the pgcopydb steam setup
command anymore, the command pgcopydb snapshot --follow must be used now.

* Fix installing default values for --plugin and --slot-name.

In particular the option --plugin also is supported via the environment
variable PGCOPYDB_OUTPUT_PLUGIN and we must refrain from overwritting the
value we might have read from there.

Then we need to adjust the tests to wait until the coproc snapshot command
has created its snapshot and files on-disk to be able to properly read the
values from there (snapshot, replication slot name and lsn, etc).

That result status comes from our implementation of a logical decoding
client, where COPY BOTH protocol is used.

We have identified the root cause of the issue where double precision values were going out of range during the COPY operation. It turns out that we were missing the extra_float_digits GUC (grand unified configuration) while copying the float8 data. As a result, when extreme values like '1.7976931348623157E+308' were copied from the source, an overflow occurred.

When switching replay operating modes we want to replay all the changes that
has been streamed locally but have not been caught-up yet. This includes
some JSON to SQL transformation work that was missed before.

To be able to receive from the transform queue after having received a
signal to stop, we need to also reset the signals control flags, otherwise
all we get is an early exit in queue_receive.

Manage the connection internally in the stream_apply_send_sync_sentinel and
stream_apply_fetch_sync_sentinel functions, closing the connection each time
we have received the result.

* Update pgcopydb sentinel's replay_lsn asynchronously.

When in follow replay mode, the pgcopydb sentinel is updated quite
frequently so that we can see progress being made and be reactive about when
to implement the migration cutover.

Now, we might have to consider some latency with the source database system
where the replay_lsn is published, so rate-limiting those sentinel updates
is important. Also, rate limiting alone might not be enough to prevent
slowing down the replay process, so switch to using libpq async API.

* Make sure to update replay_lsn before sub-process exit.

The previous dynamic memory allocation fix was shy of a brick load. See #259.

The summary commands were also statically allocated with BUFSIZE bytes,
which is not always enough to fit the index definitions etc. Apply same
approach as in commit f05d75b6.

Also add unit test coverage this time.

* Assorted streaming fixes, including skipping of empty transactions.

* Fix memory management of JSON buffers when parsing Logical Messages.

Now that we need to keep around the previous LogicalMessageMetadata, we also
need to keep around its JSON buffer. This means we can't re-use the pointer
to the current context buffer, and we need to duplicate memory even when
using wal2json and keeping the same JSON buffer as received.

In passing, adjust the JSON files in the test suite according to the new
empty transactions filtering and synthetic KEEPALIVE situation.

* Atttempt to fix CI errors (exit code 141).

The exit code 141 is 128 + 13, and 13 is SIGPIPE. It means that in our
cdc-low-level test case when doing receive | transform | apply, one of the
processes is trying to write to its output pipe when the next process has
already exited, or at least closed its pipe input file descriptor.

By ignoring SIGPIPE we ask the system to return EPIPE at write time. Let's
now see how our code handles that.

* Fix wal2json bytea values.

The wal2json output plugin processes to remove the \x prefix from Postgres
bytea strings, so pgcopydb needs to put the prefix back when transforming
from the JSON to the SQL format.

* Update unit testing files with the wal2json "type" column.

Recent changes had the impact of discarding empty transactions entrirely
from the SQL file when transforming them from the JSON file. These
transactions act somewhat like a KEEPALIVE message of sorts, as they
represent activity in the source database that is not happening in the
database we're following.

In this commit we force the transform of these empty transactions to make
them part of the SQL file.

Rather than using a static BUFSIZE string buffer we call calloc() to
allocate memory dynamically for index columns, index definitions, constraint
definition, and collation descriptions.

The allocated memory is used throughout the pgcopydb program execution, and
at the moment the memory is not de-allocated by a call to free(). Instead,
we rely on the OS to take care of that for us when exiting pgcopydb.

Instead of escaping the characters at the C code level only, issuing queries
with actual newlines in the query text, arrange to embed escaped characters
in the SQL query text (\n and \r).

It could be that the role used to connect to the source and target databases
lacks privileges to access some Postgres schema and/or objects within that
schema, which means using --exclude-schema on the command line is the right
approach.

We need to find sequences that are associated with table columns either as
default values (through pg_depend entry between the sequence and pg_attrdef)
or as an identity column (through a pg_depend entry between the sequence and
the table column directly).

Also, when a sequence is filtered-out from our pg_restore activity, then we
need to also filter-out the archive TOC entry for the default value that
uses the sequence, in addition to filtering out the table and the sequence
themselves.

  ALTER TABLE ONLY public.bar ALTER COLUMN id SET DEFAULT nextval(...)

Otherwise we get ERROR: relation does not exist in pg_restore.

See
https://www.postgresql.org/docs/current/runtime-config-logging.html#RUNTIME-CONFIG-LOGGING-JSONLOG
for the list of key names that Postgres uses when using jsonlog.