Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Log exception on inactivity callback #1194

Merged
merged 11 commits into from
May 10, 2024
Merged

Log exception on inactivity callback #1194

merged 11 commits into from
May 10, 2024

Conversation

jjanezhang
Copy link
Contributor

@jjanezhang jjanezhang commented May 10, 2024

Log exception on inactivity callback

Logs an exception on timeout so we can write a run event.

Testing

Irene tested a run test-1b-5tv00z that timed out and I confirmed that we wrote the run timeout exception to metadata:
Screenshot 2024-05-10 at 2 24 15 PM

scripts/train/train.py Outdated Show resolved Hide resolved
@jjanezhang jjanezhang self-assigned this May 10, 2024
@jjanezhang jjanezhang requested a review from dakinggg May 10, 2024 21:24
@dakinggg dakinggg enabled auto-merge (squash) May 10, 2024 21:43
@dakinggg dakinggg merged commit 994209c into main May 10, 2024
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants