Skip to content

Commit

Permalink
Properly close transactions in indexer.py
Browse files Browse the repository at this point in the history
`pg_conn.execute` opens a transaction automatically, and there are some
situations where we weren't closing it.

I made the change to capture Sentry exceptions not because I've seen issues
there in practice, but because we don't want any potential errors there to
block the subsequent transaction commit.
mwiencek committed May 22, 2024
1 parent b79a4fe commit 78c5ee2
Showing 1 changed file with 11 additions and 4 deletions.
15 changes: 11 additions & 4 deletions indexer.py
Original file line number Diff line number Diff line change
@@ -98,9 +98,10 @@ def handle_event_failure(pg_conn, event, error):
VALUES (%(event_id)s, %(error)s)
'''), {'event_id': event['id'], 'error': str(error)})

pg_conn.commit()

sentry_sdk.capture_exception(error)
try:
sentry_sdk.capture_exception(error)
except BaseException as sentry_exc:
logging.error(sentry_exc)


def cleanup_events(pg_conn):
@@ -193,7 +194,6 @@ def run_event_handler(pg_conn, event, handler):
SET state = 'completed'
WHERE id = %(event_id)s
'''), {'event_id': event['id']})
pg_conn.commit()


def indexer(
@@ -230,6 +230,9 @@ def indexer(
sleep_amount = 1
idle_loops = 0
else:
# Close the transaction opened by `get_next_event`.
pg_conn.commit()

# Since there's nothing else to do, cleanup old events
# (but not too often).
current_datetime = datetime.datetime.now()
@@ -253,6 +256,9 @@ def indexer(
continue

if event['state'] != 'queued':
# Close the transaction opened by `get_next_event`.
pg_conn.commit()

# This is mainly a development aid. In at least one
# occasion I broke the SQL query above by having bad
# boolean operator precedence. -- mwiencek
@@ -280,6 +286,7 @@ def indexer(
event,
handler,
)
pg_conn.commit()

pg_conn.close()

0 comments on commit 78c5ee2

Please sign in to comment.