Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to run CELERY_BEAT_SCHEDULE tasks at different intervals per tenant? #821

Open
royroyee opened this issue Nov 7, 2024 · 0 comments
Open

Comments

@royroyee
Copy link

royroyee commented Nov 7, 2024

Summary:

Looking for a way to stagger CELERY_BEAT_SCHEDULE tasks to run at different times per tenant, rather than all tenants executing tasks simultaneously

  • Celery Version: 5.4.0
  • Celery-Beat Version: 2.5.0

Exact steps to reproduce the issue:

  1. Define CELERY_BEAT_SCHEDULE with multiple tasks that run at regular intervals.
  2. Use this setup across multiple tenants. (django-tenants)
  3. Observe that all tenants execute tasks simultaneously, causing high resource load

Detailed information

This issue has been cross-posted on tenants-schemas-celery, but checking here as well to explore any solutions specific to django-celery-beat.

When tasks run at the same intervals across all tenants, it creates a high resource load as the number of tenants grows. A solution to vary task execution times slightly per tenant, without complex configurations or major structural changes, would be ideal.

Any guidance or examples for implementing this kind of staggered scheduling would be greatly appreciated. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant