Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Programming to Foundry #441

Merged
merged 19 commits into from
Aug 24, 2023

Conversation

rishab-partha
Copy link
Contributor

Added programming datasets and added programming to the eval yamls/models.

Copy link
Contributor

@danbider danbider left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me. @codestar12 it'll be useful for me if this got merged.

Copy link
Contributor

@bmosaicml bmosaicml left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great. Note to future self: per discussion with Rishab the official v0.1 of model gauntlet will remove human eval c and set num_fewshot to 3 for all programming tasks

@rishab-partha rishab-partha merged commit 52a3500 into mosaicml:main Aug 24, 2023
8 checks passed
rishab-partha added a commit that referenced this pull request Aug 24, 2023
dakinggg pushed a commit that referenced this pull request Aug 24, 2023
@samhavens samhavens mentioned this pull request Sep 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants