Expand options for MMLU.

[mansheej]

Modify the MMLU evaluation to expand the choices with the full string for the choice. This provides better signal for models trained at smaller FLOP counts.
Previously, a typical MMLU example looked like

{
    "query": "Question: For which of these two scenarios does the main character (who uses I/me/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I slammed on my breaks to miss the ball as it came in to the road. Scenario 2 | I taught my children to play the xylophone.\nChoices\nA. Wrong, Wrong\nB. Wrong, Not wrong\nC. Not wrong, Wrong\nD. Not wrong, Not wrong", 
    "gold": 3, 
    "choices": ["A", "B", "C", "D"], 
    "category": "moral_scenarios"
}

This example now looks like

{
    "query": "Question: For which of these two scenarios does the main character (who uses I\/me\/my) do something clearly morally wrong, according to ordinary moral standards in the US as of 2020? Scenario 1 | I slammed on my breaks to miss the ball as it came in to the road. Scenario 2 | I taught my children to play the xylophone.\nChoices\nA. Wrong, Wrong\nB. Wrong, Not wrong\nC. Not wrong, Wrong\nD. Not wrong, Not wrong",
    "gold": 3,
    "category": "moral_scenarios",
    "choices": ["A. Wrong, Wrong","B. Wrong, Not wrong","C. Not wrong, Wrong","D. Not wrong, Not wrong"]
}

the difference is in the choices.

[abhi-mosaic]

Skimmed some of the samples, trust that this was all code-generated.

I think this will be great for testing on small models and a potential candidate for Gauntlet v0.3.