Removing ∞ silent retries of Bulkrax::ImportWorkJob #692
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jeremyf
force-pushed
the
jeremyf---691
branch
from
8b3e1dd
to
3a6d657
Compare
Prior to this commit, when we encountered a `Bulkrax::CollectionsCreatedError`, we would reschedule the same job with a 1 minute delay. However, if the underlying issue was never resolved, we would continually reschedule without fanfare this failing job. With this commit, we provide 3 logged reschedules, then if that fails we raise an exception break the infinite cycle. By breaking the cycle, then the upstream Sidekiq handler can determine if it should be thrown into the `Retry` queue (where the jobs that raise exceptions go). We discovered this in Adventist watch 100 or scheduled jobs run very fast, and apparently continually rescheduling themselves. Closes #691
jeremyf
force-pushed
the
jeremyf---691
branch
from
3a6d657
to
4c390a4
Compare
bkiahstroud
approved these changes
Dec 19, 2022
jeremyf
added a commit
to scientist-softserv/adventist-dl
that referenced
this pull request
Dec 19, 2022
From the Bulkrax pull request: > Prior to this commit, when we encountered a > `Bulkrax::CollectionsCreatedError`, we would reschedule the same job > with a 1 minute delay. However, if the underlying issue was never > resolved, we would continually reschedule without fanfare this failing > job. > > With this commit, we provide 3 logged reschedules, then if that fails > we raise an exception break the infinite cycle. By breaking the > cycle, then the upstream Sidekiq handler can determine if it should be > thrown into the `Retry` queue (where the jobs that raise exceptions > go). > > We discovered this in Adventist watch 100 or scheduled jobs run very > fast, and apparently continually rescheduling themselves. Related to: - samvera/bulkrax#692 - samvera/bulkrax#691
|
as info, #610 was created for this purpose. at a later time we should handle this type of thing the same way throughout the entire app. |
4 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Prior to this commit, when we encountered a
Bulkrax::CollectionsCreatedError, we would reschedule the same job with a 1 minute delay. However, if the underlying issue was never resolved, we would continually reschedule without fanfare this failing job.With this commit, we provide 3 logged reschedules, then if that fails we raise an exception break the infinite cycle. By breaking the cycle, then the upstream Sidekiq handler can determine if it should be thrown into the
Retryqueue (where the jobs that raise exceptions go).We discovered this in Adventist watch 100 or scheduled jobs run very fast, and apparently continually rescheduling themselves.
Closes #691