Unverified Commit aea0ff08 authored by Dimitri Fontaine's avatar Dimitri Fontaine Committed by GitHub
Browse files

Implement filtering support. (#19)

* Implement a parser for filtering settings, and filter table lists.

* Apply filters to listing sequences.

We filter only those sequences that have a link dependency to the selected
tables. There is no way at the moment to select a sequence that's not
tracked to a specific default value for a column that doesn't belong to one
of the selected tables.

That said, when filtering is not used, all sequences are processed, even
those not attached to any table. Which means that `pgcopydb copy sequences`
in the use case where filtering is needed for tables, but not for sequences.

* Implement pgcopydb list ... --list-skipped.

This option allows debugging the filtering setup, and is also needed to
implement pg_restore catalog editing (--use-list) to avoid installing
objects that are filtered-out in the setup.

* Document filtering setup.

* Add a test for the filtering capabilities.

* Improve filtering of pg_restore list entries without an OID.

Some pg_restore --list entries such as INDEX ATTACH miss both the catalog
and the object oid, and then we need to match them by their pg_restore list
name, which is a compound of the schema name, object name, and owner name.

Adding to that, the way pg_restore builds that compound name is with using a
single space as a separator, and replacing \n and \r characters with a
single space. This makes the pg_restore list output unfriendly to machine
parsing, and so instead we generate the list name from the Postgres catalogs
in our catalog queries.

Then we can use an hash table on the OIDs and another one on the compound
names and find out if an INDEX ATTACH catalog entry refers to an index that
has been filtered-out by filtering rules, and comment it out then.

* Add a pg_depend recursive walker facility.

This allows filtering from pg_restore --list objects that depend on tables
that have been filtered out.
parent c8058300
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment