-
-
Notifications
You must be signed in to change notification settings - Fork 73
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unexpected close of worker after all the I/O intensive tasks has finished #84
Comments
I found a workaround today: using
this way, when you have only one worker, large number of time-consuming tasks in the message queue won't be added to the node's internal event-loop, and the heartbeat And BTW, when channel consumes the queue in the broker code, the channel acknowledges the message before the worker-registered-handler runs. Does the acknowledgement happens too soon? |
Description
I submitted 5 40-seconds-long tasks and the tasks were finished successfully causing the worker exits unexpectedly with the following exception logs:
What is the expected behavior?
I expect the worker keeps running and be able to take more tasks without crashing by above exception.
Please tell us about your environment:
16.14.0
, celery-node:0.5.8
"use strict"
andrequire()
]Other information (e.g. detailed explanation, stacktraces, related issues, suggestions how to fix, links for us to have context, eg. stackoverflow, gitter, etc)
I looked into the problem a little bit more by searching internet, and found out that it's the classic
amqplib
problem.amqplib
has a default heartbeat which is sent in a 60-seconds-long interval. My tasks did I/O intensive work that blocks the heartbeat intervals. What confuses me is that each of my task requires less than 60 seconds, and theamqplib
should have the chance to sent heartbeat in between consecutive tasks. However the fact is that heartbeat intervals got postponed to the point when all tasks have been processed. Thus, the rabbitmq detects that heartbeat is missed and closes the connection.The text was updated successfully, but these errors were encountered: