Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Using django-celery-beat， Scheduled tasks are triggered repeatedly #388

Closed
ugfly1210 opened this issue Feb 26, 2021 · 7 comments
Closed

Using django-celery-beat， Scheduled tasks are triggered repeatedly #388

ugfly1210 opened this issue Feb 26, 2021 · 7 comments

Comments

@ugfly1210
Copy link

ugfly1210 commented Feb 26, 2021 •
edited
Loading 

celery==4.4.6
django-celery-beat==2.1.0
Django==3.1.2
django-celery-results==2.0.0

Set a timing policy: execute the task at 00:10 every day task_A
look at the beat log shows that

[2021-02-26 00:10:00,052: INFO/MainProcess] Scheduler: Sending due task task_A
[2021-02-26 00:10:05,194: INFO/MainProcess] Scheduler: Sending due task task_A

celery log:

[2021-02-26 00:10:00,061: INFO/MainProcess] Received task: task_A[e30e7941-a7da-4a9f-90d9-b125f8e4c46]   expires:[2021-03-29 16:01:59.944841+00:00]
[2021-02-26 00:10:00,120: INFO/ForkPoolWorker-5] Task task_A[e30e7941-a7da-4a9f-90d9-fb125f8e4c46] succeeded in 0.05772967217490077s: True
[2021-02-26 00:10:05,233: INFO/MainProcess] Received task: task_A[c9e7ce40-664d-4bf8-a14d-84651a5885eb]   expires:[2021-03-29 16:02:05.186303+00:00]
[2021-02-26 00:10:05,298: INFO/ForkPoolWorker-5] Task task_A[c9e7ce40-664d-4bf8-a14d-84651a5885eb] succeeded in 0.06434300076216459s: True

other configuration parameters:

BROKER_BACKEND = 'redis'
CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_RESULT_SERIALIZER = 'json'
CELERY_TASK_SERIALIZER = 'json'
CELERY_TASK_RESULT_EXPIRES = 60 * 60 * 24
CELERYD_CONCURRENCY = 4  
CELERYD_MAX_TASKS_PER_CHILD = 4  
CELERYD_PREFETCH_MULTIPLIER = 4
CELERYD_FORCE_EXECV = True
# BROKER_TRANSPORT_OPTIONS = {'visibility_timeout': 90}
CELERY_CREATE_MISSING_QUEUES = True
CELERY_IGNORE_RESULT = True
CELERY_SEND_EVENTS = False
CELERY_DISABLE_RATE_LIMITS = False
# CELERY_EVENT_QUEUE_EXPIRES = 60
CELERY_TIMEZONE = TIME_ZONE
CELERY_ENABLE_UTC = False
CELERY_RESULT_BACKEND = 'django-db'
CELERY_BEAT_SCHEDULER = 'django_celery_beat.schedulers:DatabaseScheduler'
@auvipy
Copy link
Member

auvipy commented Feb 26, 2021

can you please use english

@ugfly1210 ugfly1210 changed the title 使用django-celery-beat，定时任务被重复触发 Using django-celery-beat， Scheduled tasks are triggered repeatedly Feb 28, 2021
@auvipy
Copy link
Member

auvipy commented Mar 3, 2021

can you try https://github.com/celery/django-celery-beat/releases/tag/v2.2.0 & bab79d9

@ugfly1210
Copy link
Author

can you try https://github.com/celery/django-celery-beat/releases/tag/v2.2.0 & bab79d9

Current version：
django-celery-beat==2.2.0

Still having problems with tasks triggering repeatedly= =

[2021-07-27 00:05:00,019: INFO/MainProcess] Scheduler: Sending due task generate_ss (rep.tasks.ss)
[2021-07-27 00:05:20,217: INFO/MainProcess] Scheduler: Sending due task generate_ss (rep.tasks.ss)
[2021-07-27 00:05:20,261: INFO/MainProcess] Scheduler: Sending due task generate_ss (rep.tasks.ss)

beat_loop_interval=20

The task at 00:05:00 has been executed successfully and the execution time is <20s.
At 00:05:20, the task has been generated twice more. The two different task_id

@ugfly1210
Copy link
Author

There is also a different kind of problem from the above

[2021-07-27 11:32:21,489: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2021-07-27 11:32:41,576: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2021-07-27 11:33:01,661: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2021-07-27 11:33:01,700: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2021-07-27 11:33:01,738: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2021-07-27 11:33:21,795: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2021-07-27 11:33:21,833: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2021-07-27 11:33:21,872: INFO/MainProcess] DatabaseScheduler: Schedule changed.

interval:20s
Irregularly triggered three times at the same time

@heng-zhang-20
Copy link

I also have this similar problem: beat sending task constantly, but only built-in task. This is issue link in django-celery-result project: celery/django-celery-results#275

These are logs:

[2022-01-07 01:51:27,676: INFO/Beat] Scheduler: Sending due task celery.backend_cleanup (celery.backend_cleanup)
[2022-01-07 01:51:27,684: INFO/Beat] Scheduler: Sending due task celery.backend_cleanup (celery.backend_cleanup)
[2022-01-07 01:51:27,689: INFO/ForkPoolWorker-16801] Task celery.backend_cleanup[571c20ea-012f-4cce-8778-cf83c591c6b9] succeeded in 0.022669575992040336s: None
[2022-01-07 01:51:27,690: INFO/ForkPoolWorker-16800] Task celery.backend_cleanup[73f742c1-796e-4377-9619-4fcf0d3e7e1a] succeeded in 0.012219233001815155s: None
[2022-01-07 01:51:27,691: INFO/Beat] Scheduler: Sending due task celery.backend_cleanup (celery.backend_cleanup)
[2022-01-07 01:51:27,693: INFO/MainProcess] Task celery.backend_cleanup[5370d550-161e-4d80-9309-d8e84b18fca4] received
[2022-01-07 01:51:27,696: INFO/MainProcess] Task celery.backend_cleanup[63c4e1f4-141e-4371-bff1-5e86533258d3] received
[2022-01-07 01:51:27,698: INFO/MainProcess] Task celery.backend_cleanup[5d89a5e4-faba-4814-89f0-939f8be2c4bc] received
[2022-01-07 01:51:27,700: INFO/Beat] Scheduler: Sending due task celery.backend_cleanup (celery.backend_cleanup)
[2022-01-07 01:51:27,706: INFO/Beat] Scheduler: Sending due task celery.backend_cleanup (celery.backend_cleanup)
[2022-01-07 01:51:27,708: INFO/ForkPoolWorker-16801] Task celery.backend_cleanup[4e27da29-82af-499a-83ea-13414258494b] succeeded in 0.016296524001518264s: None
[2022-01-07 01:51:27,708: INFO/ForkPoolWorker-16800] Task celery.backend_cleanup[08bc4384-6efb-4e51-ae0e-926444cc7195] succeeded in 0.017958137003006414s: None
[2022-01-07 01:51:27,713: INFO/Beat] Scheduler: Sending due task celery.backend_cleanup (celery.backend_cleanup)
[2022-01-07 01:51:27,720: INFO/Beat] Scheduler: Sending due task celery.backend_cleanup (celery.backend_cleanup)
[2022-01-07 01:51:27,726: INFO/ForkPoolWorker-16801] Task celery.backend_cleanup[8e16bf97-e377-4cdb-b52d-d3de11aac86c] succeeded in 0.015769034012919292s: None
[2022-01-07 01:51:27,728: INFO/Beat] Scheduler: Sending due task celery.backend_cleanup (celery.backend_cleanup)
[2022-01-07 01:51:27,731: INFO/ForkPoolWorker-16800] Task celery.backend_cleanup[4f656b98-0fe7-4cfd-a4e8-418ce05ecc51] succeeded in 0.01994775200728327s: None
[2022-01-07 01:51:27,733: INFO/MainProcess] Task celery.backend_cleanup[f3edf723-f43e-4e3e-81ce-c1ff6093801a] received
[2022-01-07 01:51:27,736: INFO/MainProcess] Task celery.backend_cleanup[31745008-ad0a-46fa-ba12-2bd00ad95bf1] received
[2022-01-07 01:51:27,738: INFO/MainProcess] Task celery.backend_cleanup[59cfaeff-58bf-4d85-ad08-863d4f96f912] received
[2022-01-07 01:51:27,739: INFO/MainProcess] Task celery.backend_cleanup[d0fc46d8-10c1-4c40-a644-e0561b667a49] received

@auvipy
Copy link
Member

auvipy commented Jan 12, 2022 •
edited
Loading

I also have this similar problem: beat sending task constantly, but only built-in task. This is issue link in django-celery-result project: celery/django-celery-results#275

what about celery/django-celery-results#275 (comment) ?

Django settings USE_TZ = True and set Celery config enable_utc = True, the problem goes away. It looks like a problem about time zone and utc settings.

@BaiJiangJie BaiJiangJie mentioned this issue Jul 10, 2023
Merged
@BaiJiangJie
Copy link
Contributor

Try it.

#660

auvipy pushed a commit that referenced this issue Jul 25, 2023
@BaiJiangJie
Fix Issue #388: Celery Beat scheduled tasks may be executed repeatedly (
700c804 
#660)

* Fix: The problem that Celery Beat scheduled tasks may be executed repeatedly

* fix: 增加 TypeError 捕获

* fix: Add tests for `test_sync_not_saves_last_run_at_while_schedule_changed`
@auvipy auvipy closed this as completed Jul 25, 2023
@alexgmin alexgmin mentioned this issue Apr 25, 2024
Open
18 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@auvipy @ugfly1210 @BaiJiangJie @heng-zhang-20