RQ Job Timeout Handling #223
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Always close the temp file on exception handling. - Added handling for the rq job timeout.
ThrawnCA
reviewed
Aug 2, 2024
- Moved try/catch for `tabulator_load` rq timeout up.
ThrawnCA
reviewed
Aug 16, 2024
|
@JVickery-TBS i'd like the idea but with the tests failing, I can't look at merging this. Also unsure about what @ThrawnCA picked up on another path that is not caught correctly. |
|
Hi @JVickery-TBS , I'd like to incorperate this into xloader. Are you able to rebase to master so that we have the tests fixed. Regards, |
|
@duttonw hey! SORRY! I have been swamped with work and general life stuff. (But I did just get married so yay!) Hoping to get back into all this XLoader and Validation stuff in the coming weeks. |
# Conflicts: # ckanext/xloader/jobs.py ### RESOLVED.
- Changed positioning of try/catch for job timeout to handle all of the type guessing retries.
- Log level for `ckanext.xloader`
- No alias import to fix try/catch.
- TMPDIR fixes.
- Simplify test with specific file suffix check.
- Typo in logging.
duttonw
reviewed
Dec 9, 2024
duttonw
reviewed
Dec 9, 2024
- More attempts...
|
@duttonw lemme know if this one is good, and I can make a squashed version |
- Added to change log.
JVickery-TBS
added a commit
to JVickery-TBS/ckanext-xloader
that referenced
this pull request
Dec 9, 2024
- Squash of ckan#223
ThrawnCA
reviewed
Dec 9, 2024
| except FileNotFoundError: | ||
| pass | ||
| logger.warning('Job timed out after %ss', RETRIED_JOB_TIMEOUT) | ||
| raise JobError('Job timed out after {}s'.format(RETRIED_JOB_TIMEOUT)) |
There was a problem hiding this comment.
If you're generating this string anyway, might as well use it for the logger as well.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
fix(logic): close temp file on rq timeout;