Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix several Postgresql errors #238

Merged
merged 1 commit into from
Jun 22, 2021
Merged

Fix several Postgresql errors #238

merged 1 commit into from
Jun 22, 2021

Conversation

hannahhoward
Copy link
Contributor

Goals

Solve running dealbot issues that reference pq -- i.e. our postgresql library

Implementation

  • setup a test harness that allows us to run our state tests with either SQLite or PostgreSQL. The default remains SQLite, but if the tests are run with POSTGRES_BIN env var set (a bin dir for PG's various executables), then each test will make a temporary test PostgresSQL cluster and execute the tests with a PG connector. this allows us to test locally at least for PG when adding state features
  • Also add a state task for draining + completing workers
  • Now solve the various issues with PG:
    • Concurrent assign task calls were not working cause we weren't using serializable transactions
    • But at the same time, our PG RetryableError method had an issue that needed fixing
    • Fix the query with oldestAvailableTaskWithTagsSQL for PG that is causing 500's on popTask on the dealbot
    • Fix ResetWorkerTasks and Complete for PSQL which has to do with the fact that row queries must be complete before doing further operations while in a transaction (this is potentially somewhat memory inefficient -- we can certainly improve it with PSQL cursors if it becomes an issue, but for now, probably best to just do the simple thing)
      Add test harness for PSQL and verify all database operations work with PSQL

Add test harness for PSQL and verify all database operations work with PSQL
@hannahhoward hannahhoward requested a review from gammazero June 22, 2021 22:03
@willscott willscott merged commit fa4ac18 into main Jun 22, 2021
@willscott willscott deleted the fix/psql_errors branch June 22, 2021 22:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants