Needs a recursive query in schema.c to reach from the table to the column to
its data type, etc.

When editing the schema, the foreign keys that point from selected tables
are only half the story.

Add a facility to dump DDL commands for a specific set of tables, and
include also their indexes and the foreign key definitions that point to
selected tables. The command is pgcopydb dump sql-files --filters ...

Each SQL object definition is saved in their own file: <oid>.sql

Later implement a way to use the saved files instead of the pg_dump archive
contents when preparing the schema on the target database, in a way that if
the SQL files have been edited the new schema is installed there.

See #275 

This requires loading more of the Postgres catalog information in-memory.
That's the cost to fix extension configuration tables that have changed
schema between source and target, and to later support schema changes at
pgcopydb time.

When the previous streaming mode was REPLAY and we attempt to empty the
transform queue using `stream_transform_from_queue`, the function waits
indefinitely for `QMSG_TYPE_STOP` (or any message) because the queue has
already been emptied.

It turns out that we can't just call pg_create_logical_replication_slot() in
a transaction that uses a snapshot to sync the initial copy of the data with
the change data capture. We need to use the CREATE_REPLICATION_SLOT logical
replication command with the RESERVE_WAL true SNAPSHOT 'export' options.

* Add a new test case that fails because of missing data around replication slot creation.

* Implement tracking of --plugin, --origin, and --slot-name options.

We already tracked --snapshot so that we can --resume and continue operating
with the same snapshot as before, and we should really have the same
facility for the rest of the replication based options.

* In --follow mode, export the snapshot via CREATE_REPLICATION_SLOT.

The previous code would set transaction snapshot to a previously exported
snapshot and then create the logical replication slot, but that creates a
replication gap. Postgres supports only one way to avoid that gap, which
consists of exporting the snapshot used to create the replication slot.

In passing, register the full information we get from creating the
replication slot to file, in such a way that we can re-use that file and
information later, including the consistent_point LSN that is associated
with the replication slot... and the snapshot.

As a result, the following commands have been removed from the CLI:

    pgcopydb stream create slot
    pgcopydb stream create origin
    pgcopydb stream drop slot
    pgcopydb stream drop origin

The only way to create a replication slot with pgcopydb is now the command
`pgcopydb snapshot --follow`, and the replication origin on the target must
be created with the consistent_point LSN returned by the same command.

* Fix unit tests to adjust to the new API/CLI.

The creation of a replication slot is not done in the pgcopydb steam setup
command anymore, the command pgcopydb snapshot --follow must be used now.

* Fix installing default values for --plugin and --slot-name.

In particular the option --plugin also is supported via the environment
variable PGCOPYDB_OUTPUT_PLUGIN and we must refrain from overwritting the
value we might have read from there.

Then we need to adjust the tests to wait until the coproc snapshot command
has created its snapshot and files on-disk to be able to properly read the
values from there (snapshot, replication slot name and lsn, etc).

That result status comes from our implementation of a logical decoding
client, where COPY BOTH protocol is used.

We have identified the root cause of the issue where double precision values were going out of range during the COPY operation. It turns out that we were missing the extra_float_digits GUC (grand unified configuration) while copying the float8 data. As a result, when extreme values like '1.7976931348623157E+308' were copied from the source, an overflow occurred.

When switching replay operating modes we want to replay all the changes that
has been streamed locally but have not been caught-up yet. This includes
some JSON to SQL transformation work that was missed before.

To be able to receive from the transform queue after having received a
signal to stop, we need to also reset the signals control flags, otherwise
all we get is an early exit in queue_receive.

Manage the connection internally in the stream_apply_send_sync_sentinel and
stream_apply_fetch_sync_sentinel functions, closing the connection each time
we have received the result.

* Update pgcopydb sentinel's replay_lsn asynchronously.

When in follow replay mode, the pgcopydb sentinel is updated quite
frequently so that we can see progress being made and be reactive about when
to implement the migration cutover.

Now, we might have to consider some latency with the source database system
where the replay_lsn is published, so rate-limiting those sentinel updates
is important. Also, rate limiting alone might not be enough to prevent
slowing down the replay process, so switch to using libpq async API.

* Make sure to update replay_lsn before sub-process exit.

The previous dynamic memory allocation fix was shy of a brick load. See #259.

The summary commands were also statically allocated with BUFSIZE bytes,
which is not always enough to fit the index definitions etc. Apply same
approach as in commit f05d75b6.

Also add unit test coverage this time.

* Assorted streaming fixes, including skipping of empty transactions.

* Fix memory management of JSON buffers when parsing Logical Messages.

Now that we need to keep around the previous LogicalMessageMetadata, we also
need to keep around its JSON buffer. This means we can't re-use the pointer
to the current context buffer, and we need to duplicate memory even when
using wal2json and keeping the same JSON buffer as received.

In passing, adjust the JSON files in the test suite according to the new
empty transactions filtering and synthetic KEEPALIVE situation.

* Atttempt to fix CI errors (exit code 141).

The exit code 141 is 128 + 13, and 13 is SIGPIPE. It means that in our
cdc-low-level test case when doing receive | transform | apply, one of the
processes is trying to write to its output pipe when the next process has
already exited, or at least closed its pipe input file descriptor.

By ignoring SIGPIPE we ask the system to return EPIPE at write time. Let's
now see how our code handles that.

* Fix wal2json bytea values.

The wal2json output plugin processes to remove the \x prefix from Postgres
bytea strings, so pgcopydb needs to put the prefix back when transforming
from the JSON to the SQL format.

* Update unit testing files with the wal2json "type" column.

Recent changes had the impact of discarding empty transactions entrirely
from the SQL file when transforming them from the JSON file. These
transactions act somewhat like a KEEPALIVE message of sorts, as they
represent activity in the source database that is not happening in the
database we're following.

In this commit we force the transform of these empty transactions to make
them part of the SQL file.

Rather than using a static BUFSIZE string buffer we call calloc() to
allocate memory dynamically for index columns, index definitions, constraint
definition, and collation descriptions.

The allocated memory is used throughout the pgcopydb program execution, and
at the moment the memory is not de-allocated by a call to free(). Instead,
we rely on the OS to take care of that for us when exiting pgcopydb.

Instead of escaping the characters at the C code level only, issuing queries
with actual newlines in the query text, arrange to embed escaped characters
in the SQL query text (\n and \r).

It could be that the role used to connect to the source and target databases
lacks privileges to access some Postgres schema and/or objects within that
schema, which means using --exclude-schema on the command line is the right
approach.

We need to find sequences that are associated with table columns either as
default values (through pg_depend entry between the sequence and pg_attrdef)
or as an identity column (through a pg_depend entry between the sequence and
the table column directly).

Also, when a sequence is filtered-out from our pg_restore activity, then we
need to also filter-out the archive TOC entry for the default value that
uses the sequence, in addition to filtering out the table and the sequence
themselves.

  ALTER TABLE ONLY public.bar ALTER COLUMN id SET DEFAULT nextval(...)

Otherwise we get ERROR: relation does not exist in pg_restore.

See
https://www.postgresql.org/docs/current/runtime-config-logging.html#RUNTIME-CONFIG-LOGGING-JSONLOG
for the list of key names that Postgres uses when using jsonlog.

This allows pgcopydb to log both to the terminal and also to a file, where
the format is not adjusted for terminal width (long time format, source file
and line always included).

Also implements logging in JSON, using a JSON object per line with a static
set of attributes that includes all the metadata we have about the log
message.

The control of this feature set is entirely based on using environment
variables, no new command line switch has been introduced. It is possible to
log both in plain text with colors on the terminal and to a JSON file at the
same time.

The --verbose increments the logLevel from “normal” to verbose (or notice),
then SQL, then DEBUG, then TRACE. Previous coding would skip the SQL level
when incrementing the level, this fixes it.

This allows postponing the first round of VACUUM ANALYZE on the target
tables after enabling the production workload again.

Rather than having to add so many parameters just re-use the options struct
as a parameter there.

We still want to have a separate command line option structure rather than
use the internal tracking CopyDataSpec structure at command line parsing
time, but we can still simplify the code a good deal.

This allows easier adding of new command line options in the future.

Careful reading of the Explicit Locking chapter of Postgres documentation
shows that VACUUM and ANALYZE take a SHARE UPDATE EXCLUSIVE lock against the
target table, and CREATE INDEX take a SHARE lock, which conflicts with the
former one.

  https://www.postgresql.org/docs/current/explicit-locking.html

So we can't actually VACUUM ANALYZE a table while building the indexes and
then constraints on top of that. The queueing of the VACUUM operations is
now done after the indexes and constraints are built.

In the apply process logging the SQL statements opens the door to logging
user data. For privacy concerns, reduce the apply module logging.

Also reduce the amount of logs sent by the receive and apply processes in
general, as this can lead to a very large volume of data and we do not
anciticape needing that level of details to be able to debug operations.