TID scan might not be supported by other Table AM (such as Citus Columnar).

The Postgres system column "ctid" is the physical location of the row
version within its table. It is encoded using the tid datatype:
`(pagenumber,rownumber)`.

Every single Postgres table has a "ctid" column and it is always possible to
split a table's contents by using ctid based ranges. The question of this
approach being good at reducing pgcopydb timings remains open: this PR is
meant for allowing more experimentations.

Introduce a new C module "compare.c" and implement the work there, leaving
the "cli_compare.c" module with command line and output handling.

In passing, add the checksum information to the schema.json file at the end
of the pgcopydb compare data command, that might be useful.

That way pgcopydb compare data waits for as long as the slowest query that
are running concurrently on source and target instances, which is better
than serial execution on two different servers.

This changes the JSON format output, because we also now skip fetching the
target database catalogs.

First, the parsing of the schema and table names from the INI file was done
wrong, in a way that shows with long schema and table names.

Then, I just learned we can't use pg_restore --schema in our context,
because it would then skip the CREATE SCHEMA statement and pgcopydb relies
on pg_restore to create the schema on the target database,

* Implement concurrent workers for Large Objects data copy.

The default is 4 workers for Large Object data, and another process is
created to queue the Large Object metadata (oid) and allow workers to share
the workload. New option available: --large-objects-jobs.

* Fix blob summary.

Reinstall a blob summary file and change its format to JSON.

Use an output format that is stable in number of digits and can deal with a
bigint overflow (in Postgres a sum(bigint) is numeric): use an MD5 sum and
represent it as an UUID.

Also, include the row count in the MD5 computation to better protect against
collisions.

Finally, add support for pgcopydb compare data --json.

Implement a set of pgcopydb commands to compare databases, so that it's
possible to make sense of a success criteria for the migration. The current
implementation is pretty limited and provides a starting point in that
direction.

Despite the name, this filter is an exclusion filter and works exactly the
same as the existing "exclude-schema" filter. The new facility is syntactic
sugar and allows listing schemas to keep around, which is useful when
keeping a single schema and excluding the rest of them.

Instead of using a static buffer of BUFSIZE for building the SQL string, use
the PQExpBuffer API instead here too.

This option allows specifying which version of an extension to install on
the target database. The version might be different from the version on the
source, and in some cases installing the default version of an extension is
not desired. Examples are PostGIS v2 and v3 series, for instance.

A requirement file in JSON can be given to pgcopydb commands. The JSON file
should be an array of objects with the "name" and "version" keys, and can be
given to either of these commands:

    pgcopydb copy extensions --requirements requirements.json
    pgcopydb clone ... --requirements requirements.json

To list the available extension versions on the target database use the
following commands:

    pgcopydb list extension --available-versions
    pgcopydb list extension --available-versions --json

    pgcopydb list extension --requirements
    pgcopydb list extension --requirements --json

The last command will output all the extensions available on the target
database with their default version. It's then possible to edit the output
file and change the version numbers before giving it back to the copy and
clone commands.

A single sequence could be used by more than one table/column thanks to
calling the nextval() function in the default value definition of the
column. That said, a sequence is OWNED BY a single table.

In this patch we fix the case where the table that owns the sequence is
filtered out from the pgcopydb scope while another table that uses the same
sequence is to be copied over... with the sequence, then.

A simple way to create the situation is using the following DDL:

    create table app.foo(id bigserial, f1 text);
    create table copy.foo(like app.foo including all);

And then filter-out schema app from pgcopydb scope.

This resolves a problem encountered during the transition from prefetch to
replay mode, where the transform process identifies some incomplete
transactions. This issue also arises when we interrupt 'follow' and execute
a 'follow --resume'.

The root cause is upon resumption of streaming, we may encounter a
transaction where the initial messages have LSNs less than the startpos
calculated from the latest json file.

The implemented fix involves comparing the LSNs with 'startpos' only when
resuming streaming of partially complete transactions. Once 'startpos' is
reached, comparison of LSNs for subsequent messages is discontinued.

Fixes: #373

Should fix #317.

First, when interrupting the pgcopydb process interactively with C-c while
the transform process is running, we should exit the follow loop cleanup.

Second, when using pgcopydb clone --follow --restart we should do a
replication setup cleanup first, the same as the pgcopydb stream cleanup
command, so that we can actually restart from scratch as the command says.

First, Postgres type names might contain square brackets (arrays), which are
used as a delimited in the test_decoding format for typenames. Fix the code
to handle the following properly:

    f2[text[]]:'{incididunt,ut,labore,et,dolore,magna}'

Then, it turns out that test_decoding emits text in standard SQL form, which
is to say in single-quotes and with single-quotes doubled, but with the \n
and other characters not-escaped. We need to edit the representation a
little so that the ld_transform module processes the text correctly.

See #323.
Fixes #377.

The new commands allow listing extension versions in different ways and
formats (human-friendly table, JSON):

  $ pgcopydb list extensions --available-versions
  $ pgcopydb list extensions --requirements

The idea is to prepare for a feature where the extension versions to install
on the target database can be provided by the user. To prepare for that, the
new commands allow listing available extension versions on the target
database and preparing the current requirements file from the extensions
available on the target and their default version.

The connection strings are now dynamically allocated but cli_stream somehow
didn't get the memo so far.

* Implement multi-parts read from Unix PIPE.

A Unix PIPE is typically 64kB large and we might have JSON messages that
won't fit in that. The size of the buffer filled by the read() call used to
be fixed at 128MB, but that doesn't have an impact on how much data we can
receive in a single read() call actually.

Fix that properly and allocate the size of the PIPE buffer thanks to a call
to fnctl(fd, FIONREAD) which seems to be well supported. Then use the libpq
string buffer facility to build the messages in memory when receiving them a
piece at a time from the Unix PIPE buffer, in multiple read() calls.

* Ensure our read_from_stream buffer terminates properly.

C-string must be terminated by '\0' and the read() call is not ensuring
that.

The list command does not use a --target connection string, so we skip
preparing the target catalogs in that case. In passing, also fix a strange
way that PQgetisnull() was done for sequences, as the misleading error meant
this code got a free round of review.

* Review double-precision format string from %g to %f.

That's apparently how to write double-precision integer values without
loosing precision.

* Use fmod() to determine if the float value is an integer.

Implement a retry loop in case the PIPE buffer is full or something else
requires several write attempts before succeeding to write the whole buffer
there.

* Fix skipping streaming messages when resuming from latest JSON file.

It might happen that several JSON messages have the same LSN in our stream,
usually that concerns KEEPALIVE and BEGIN and COMMIT message. Unless paying
attention to the latest message action, we could skip a BEGIN message that
has the same LSN as the latest message even if that message was a KEEPALIVE.

Fix this by keeping around the latest message action. It should be all we
need to distinguish a new message from the latest one.

* Registering latest message at resume properly.

It could be that we have several messages with the same LSN: I, L, E, for
instance. Take that into account in streamCheckResumePosition.

* Code review.

The return value should be about the success of the operation, not about the
fact that the apply mode is enabled or disable: that information is tracked
properly in the StreamApplyContext instance given as a parameter.

We probably should parse the JSON file as soon as it exists rather than only
when a SQL file was already produced in the previous round.

On Azure Flexible Server for Postgres the COMMENT ON EXTENSION command is
not supported as a non-superuser and could cause migrations to fail. Waiting
for a fix in Azure, allow skipping that part of the pg_dump and restore
activity.

When resuming operations from pre-existing files on-disk, we implement some
cache invalidation by transforming the JSON file again from scratch, and
tracking progress in our internal data structures.

That's needed in case the previous version of the file terminates in the
middle of a transaction rather than at a clean transaction boundary.

The previous approach missed that the internal tracking state kept in the
StreamContext instance was not shared. Now it is.