Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use more than one db connection #21

Closed
madadam opened this issue Jan 24, 2022 · 3 comments
Closed

Use more than one db connection #21

madadam opened this issue Jan 24, 2022 · 3 comments
Assignees

Comments

@madadam
Copy link
Collaborator

madadam commented Jan 24, 2022

Currently limiting the db connections to one to avoid the "database table is locked" errors when multiple connections access the same table simultaneously. This seems to be a limitation of the sqlx crate. Issue raised upstream: launchbadge/sqlx#1649

EDIT: with the shared cache disabled we are now facing different issues which forced us to go back to single db connections:

  • Without shared cache, it's impossible to use memory databases with multiple connections as each connection would create its own separate db. This means we would have to change our testing strategy. One option is to use temporary files, but we need to be careful that we properly delete them at the end of the tests and that we don't delete them prematurely (which would lead to "Disk I/O" errors). NOTE: we could also use single connection + memory db in tests and multiple connection + file db in production. That's not ideal as the test diverges a bit too much from the production and it could miss bugs.

  • With file databases and multiple connections I was still experiencing SQLITE_BUSY errors even with the busy timeout set. After some experimentation I found out that setting the synchronous pragma to NORMAL (default is FULL) seemed to have fixed those errors. It's unclear why that is the case and also whether it's really a fix or it just made the problem less likely to occur. It needs more investigation.

@madadam madadam self-assigned this Jan 25, 2022
@madadam
Copy link
Collaborator Author

madadam commented Jan 25, 2022

One option to do this without the upstream change is to use databases in temporary files instead of in memory. That would avoid shared cache which is what causes the table locked errors.

@madadam
Copy link
Collaborator Author

madadam commented Jan 27, 2022

Upstream PR created: launchbadge/sqlx#1658

@madadam madadam closed this as completed in a8b1d46 Feb 1, 2022
@madadam
Copy link
Collaborator Author

madadam commented Feb 2, 2022

Reopening because the applied fix turned out insufficient and we reverted back to using only one connection.

@madadam madadam reopened this Feb 2, 2022
@madadam madadam removed their assignment Apr 25, 2022
@madadam madadam self-assigned this Aug 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant