calibration standards for French prompt sets

[rogthefrog]

• Generated calibration standards using the new official fr_fr prompt sets
• Added calibration standards for the demo prompt set

[github-actions]

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

[wpietri]

Seems a little odd to me that the practice calibration all have only two significant digits. And, come to think of it, that the demo have three. Shouldn't it be the other way around?

Also, for consistency with other uids, should we really have the duplicate fr_fr in these UIDs? I would expect them to be more like safe_hazard-1.0-dfm-fr_fr-practice.

[rogthefrog]

We support multiple locales per prompt set. So a (locale, prompt-set-type) pair isn't sufficient to uniquely identify a prompt set.

[rogthefrog]

Re. the significant digits, I did have the labels backwards.

I'll rerun the calibration with the demo set and update once they're done.

[wpietri]

What's an example of having multiple locales in a prompt set? I was thinking they'd be separate.

And thanks for the update. FYI, there's no need to calibrate specifically for the demo set; the conclusion from Kurt in today's meeting was that the demo set was too small for statistical comfort, so we should just use the calibration for the set they're drawn from. @dhosterman was going to add code for that.

[rogthefrog]

What's an example of having multiple locales in a prompt set? I was thinking they'd be separate.

Currently, the prompt set files contain one locale, but the code and associated design comments indicate there may be more than one locale per prompt set file.

modelbench/src/modelgauge/tests/safe_v1.py

Line 58 in 6046471

- There many be multiple personas and locales in one file.

If we want to get rid of that support, I'm all for it, because it would simplify things a lot.