Add a timeout to the workflow API #937
Open
+38
−5
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
backend
schemas
javiermtorres
force-pushed
the
issue-934-allow-configurable-timeout
branch
from
6cbb975 to
96b3c55
Compare
njbrake
reviewed
Feb 21, 2025
njbrake
reviewed
Feb 21, 2025
Comment on lines
+63
to
+72
| # Maybe move to the job service? | ||
| def _stop_job(self, job_id: UUID): | ||
| resp = requests.post(urljoin(settings.RAY_JOBS_URL, f"{job_id}/stop"), timeout=5) # 5 seconds | ||
| if resp.status_code == HTTPStatus.NOT_FOUND: | ||
| raise JobUpstreamError("ray", "job_id not found when retrieving logs") from None | ||
| elif resp.status_code != HTTPStatus.OK: | ||
| raise JobUpstreamError( | ||
| "ray", | ||
| f"Unexpected status code getting job logs: {resp.status_code}, error: {resp.text or ''}", | ||
| ) from None |
There was a problem hiding this comment.
Hmm. Yeah this does seem like something that should be in the job service.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What's changing
Jobs in a workflow currently expire at a fixed timeout of 600 seconds. Also, timed out jobs are not stopped. This PR provides a timeout field in the workflow that sets the waiting time for all jobs in the workflow, with the expected default value of 600 seconds. It should also stop jobs that have timed out.
Closes #934
How to test it
Steps to test the changes:
Additional notes for reviewers
N/A
I already...
/docs)