Skip to content

Support celerycam for reconnecting db connenction if db connection is lost accidentally #550

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

jxltom
Copy link
Contributor

@jxltom jxltom commented Jul 6, 2018

Currently if the database is not stable and connection is lost, celerycam can not reconnect db connection automatically and it can not work properly anymore. The error connection already closed will always be raised.

This PR supports to automatically reconnect db connection for celerycam if db connection is lost.

@auvipy auvipy requested a review from Copilot May 13, 2025 09:09
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Adds automatic reconnection logic for Celery snapshot state handling when the Django database connection is lost.

  • Imports connection and InterfaceError from Django.
  • Wraps worker and task handling in a try/except to catch a closed connection.
  • Closes the connection and logs a message so Django’s connection pooling can reconnect.
Comments suppressed due to low confidence (1)

djcelery/snapshot.py:141

  • Ensure that a logger instance is defined or imported in this module before using logger.info(), otherwise this will raise a NameError at runtime.
logger.info(

except InterfaceError as e:
# When connection already closed exception is raised,
# force to close connection and Django will automatically reconnect
if str(e) == 'connection already closed':
Copy link
Preview

Copilot AI May 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comparing exception message strings is brittle; consider catching a more specific exception subclass or inspecting error attributes instead of matching on str(e).

Suggested change
if str(e) == 'connection already closed':
if getattr(e, 'args', None) and 'connection already closed' in e.args[0]:

Copilot uses AI. Check for mistakes.

Comment on lines +132 to +140
try:
for worker in state.workers.items():
self.handle_worker(worker)
_handle_tasks()
except InterfaceError as e:
# When connection already closed exception is raised,
# force to close connection and Django will automatically reconnect
if str(e) == 'connection already closed':
connection.close()
Copy link
Preview

Copilot AI May 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The try/except block wraps both worker and task handling, which may swallow unrelated errors. Consider narrowing the scope to only the database operations that need reconnection.

Suggested change
try:
for worker in state.workers.items():
self.handle_worker(worker)
_handle_tasks()
except InterfaceError as e:
# When connection already closed exception is raised,
# force to close connection and Django will automatically reconnect
if str(e) == 'connection already closed':
connection.close()
for worker in state.workers.items():
self.handle_worker(worker)
_handle_tasks()
try:
# Check and handle database reconnection if needed
connection.close()
except InterfaceError as e:
# When connection already closed exception is raised,
# force to close connection and Django will automatically reconnect
if str(e) == 'connection already closed':

Copilot uses AI. Check for mistakes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant