feat(postgres): use faster sql.DB instead of docker exec psql for snapshot/restore #2599
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
The postgres module has functions for saving snapshots and restoring them later.
This feature is helpful for integration testing without having to manually clear the database.
This PR speeds up the snapshot/restore functionality by executing the necessary SQL commands via a regular
sql.DB
connection.If the driver is not available in the test environment, or the commands fail for some other reason (SSL config, etc), it falls back to the old variant of calling
docker exec
.Why is it important?
docker exec
can be unnecessarily slow, especially when using docker in a VM with Docker Desktop or colima.On my laptop (MacBook Air M1), each call to
Container.Restore()
takes about 150~200ms.In my suite of 20 tests, where each test restores the database before starting, with creating the initial container, this brings the total time up to 12 seconds, which is quite a lot for a few small database tests.
Related issues
How to test this PR
Follow-ups
I've also considered caching the DB pool that is created to manage the databases.
After running some benchmark with this code (from branch X), I noticed it does not have a big impact on the whole test suite - my internal test suite does not change in execution time (since the time is dominated by re-connecting the actual test client, which will be terminated when the database gets deleted during the restore).
The module/postgres test suite even gets slower:
cfstras@9ad456d: