- Jan 13, 2022
-
-
Dimitri Fontaine authored
-
Dimitri Fontaine authored
-
Dimitri Fontaine authored
-
Dimitri Fontaine authored
-
- Jan 12, 2022
-
-
Dimitri Fontaine authored
-
Dimitri Fontaine authored
This allows using pg_restore --clean --if-exists, which drops target objects before creating them again. This is useful to run the pgcopydb command again and again.
-
Dimitri Fontaine authored
The estimated row counts found in pg_catalog.pg_class.reltuples is only meaningful when ANALYZE was run recently enough, or at all, otherwise all the tables are reporting zero there. Instead, let's use pg_table_size() result. It's not that expensive to get.
-
Dimitri Fontaine authored
The new commands make more sense now. We got rid of the non-implemented per-table operation modes that aren't interesting anyway, and implement actually interesting sub-commands now with the schema manipulation bits. pgcopydb copy-db Copy an entire database from source to target + dump Dump database objects from a Postgres instance + restore Restore database objects into a Postgres instance + list List database objects from a Postgres instance help print help message version print pgcopydb version pgcopydb dump schema Dump source database schema as custom files in target directory pre-data Dump source database pre-data schema as custom files in target directory post-data Dump source database post-data schema as custom files in target directory pgcopydb restore schema Restore a database schema from custom files to target database pre-data Restore a database pre-data schema from custom file to target database post-data Restore a database post-data schema from custom file to target database pgcopydb list tables List all the source tables to copy data from indexes List all the indexes to create again after copying the data
-
- Jan 11, 2022
-
-
Dimitri Fontaine authored
Otherwise we can't run the tests again, because the Postgres target instance still have the schema/data from the previous run.
-
- Jan 10, 2022
-
-
Dimitri Fontaine authored
-
- Jan 08, 2022
-
-
Dimitri Fontaine authored
This is a first step towards automated testing. Two Postgres instances are deployed, the pagila database is deployed in the source instance and then copied over to the target instance using pgcopydb. The next step consists of using a Postgres schema diff tool to check that the source and target schemas are indeed the same.
-
Dimitri Fontaine authored
This is a classic copy/paste bug where the wrong variable name is used, and because getopt makes the variable a global (well, I think that's what is happening here) it compiles clean anyway. Oops. Found while preparing some tests, which shows how much we need them...
-
Dimitri Fontaine authored
This allows building pgcopydb for debian, and the commit includes some clean-up needed for Linux compatibility (missing headers, extra warnings).
-
- Jan 07, 2022
-
-
Dimitri Fontaine authored
-
Dimitri Fontaine authored
Makes it easier to read and makes the env support possible to discover while waiting for the proper documentation and manual pages to show up.
-
Dimitri Fontaine authored
This will allow better Docker integration down the road, with full control from the docker-compose file or from a Docker environment file. Also could turn to be useful for direct usage too.
-
Dimitri Fontaine authored
This allows controling the concurrency of CREATE INDEX operations separately from the concurrency of the COPY operations. Also, the index operations are limited globally, not per-table. After all it makes sense for the users to setup how many CPUs they want to be active creating indexes at the same time globally.
-
- Jan 06, 2022
-
-
Dimitri Fontaine authored
-
Dimitri Fontaine authored
-
Dimitri Fontaine authored
-
Dimitri Fontaine authored
-
Dimitri Fontaine authored
-
- Jan 05, 2022
-
-
Dimitri Fontaine authored
-
Dimitri Fontaine authored
This way we can collect statistics at the end of the pgcopydb operation.
-
Dimitri Fontaine authored
Centralize decision making for filenames and paths that are used all along the multiple operations and stages, so that it's easier to review and reason about.
-
Dimitri Fontaine authored
When using the `pgcopydb dump db` command, and when the target directory does not exist yet, we should create it now, with all the sub-directories.
-
- Jan 04, 2022
-
-
Dimitri Fontaine authored
Also we now remove the lockfiles when done with the process itself. This should make it easier to diagnose what's happening in the running process from the outside.
-
Dimitri Fontaine authored
-
- Jan 03, 2022
-
-
Dimitri Fontaine authored
-
Dimitri Fontaine authored
-
- Dec 24, 2021
-
-
Dimitri Fontaine authored
-
Dimitri Fontaine authored
-
Dimitri Fontaine authored
-
Dimitri Fontaine authored
This requires parsing the pg_restore --list output and filtering out those SQL objects that we have created already in the parallel create index and constraints step.
-
Dimitri Fontaine authored
-
Dimitri Fontaine authored
At the moment the index jobs limit is not taken into account, we just fork as many sub-processes as we have to create indexes for each table. We also create constraints attached to indexes from their definitions, once the index is ready. This allows even primary key indexes to be built in parallel to the other indexes on a same table.
-
Dimitri Fontaine authored
It's far simpler to use the COPY protocol here compared to maybe running subprocesses with a pipe for pg_dump | pg_restore as initially planned. Also the buffering is all up to Postgres' copy API, so the code is still pretty simple overall.
-
- Dec 23, 2021
-
-
Dimitri Fontaine authored
-
Dimitri Fontaine authored
-
- Dec 22, 2021
-
-
Dimitri Fontaine authored
-