Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I've noticed big harakiri spikes in my carbon stats and after checking logs it turns out it's just one worker pid getting many SIGKILLs:
Problem is that if process hangs and harakiri tries to kill it, it doesn't need to die instantly, AFAIK it can be running for a while if it waits for I/O. So I've added
worker[nr].pending_harakiri
, if this has non-zero value than worker is being killed.worker[nr].harakiri_count
is only incremented ifpending_harakiri == 0
Since
worker[nr].pending_harakiri
stores number of SIGKILLs Number of SIGKILLs worker recived it is also logged for debbuging: