Skip to content

Bugfix/fix eval notebooks #93

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
pradyumna-rfai merged 5 commits into from
Nov 13, 2025
Merged

Bugfix/fix eval notebooks #93

pradyumna-rfai merged 5 commits into from
Nov 13, 2025

Conversation

@pradyumna-rfai
Copy link
Collaborator

@pradyumna-rfai pradyumna-rfai commented Nov 13, 2025

Changes

Fixes the following bugs -

  • "Rate limit exceeded" error when printing aggregated results
  • "No CUDA GPUs are available" error for gsm8k evals notebook
  • "Failed to unpickle serialized exception" for sci-fact evals notebook
  • "No experiment currently running" message for experiment.end() call.

Testing

  • Ran gsm8k evals notebook E2E on 2-GPU A6000
  • Ran sci-fact evals notebook E2E on 2-GPU A6000
  • Ran fiqa evals notebook E2E on 2-GPU A6000

Screenshots

Screenshot 2025-11-12 at 11 41 41 PM Screenshot 2025-11-12 at 11 43 15 PM

david-rfai
david-rfai approved these changes Nov 13, 2025
View reviewed changes
Copy link
Collaborator

@david-rfai david-rfai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@pradyumna-rfai pradyumna-rfai merged commit 0321133 into main Nov 13, 2025
@pradyumna-rfai pradyumna-rfai deleted the bugfix/fix-eval-notebooks branch November 13, 2025 20:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@david-rfai david-rfai david-rfai approved these changes

@arun-rfai arun-rfai Awaiting requested review from arun-rfai

@humaira-rf humaira-rf Awaiting requested review from humaira-rf

@anay-rfai anay-rfai Awaiting requested review from anay-rfai

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@pradyumna-rfai @david-rfai