Help Needed with Local Dataset Support for PPL Evaluation

[TianYaDY]

Describe the bug

First off, thanks a lot for all your hard work on the project. It's really helpful! I ran into something that I'm not sure about—I noticed that the PPL evaluation seems to only work with datasets like wikitext, ptb, and ptb_new. I couldn't find a way to use it with a local dataset. Is this a bug, or maybe a feature that's not added yet?

If you could clarify this for me, I would really appreciate it. Thanks again for your awesome work!

Is there an existing issue for this?

• I have searched the existing issues

Reproduction

Perplexity evaluation sub-tab of the Training tab

Screenshot

No response

Logs

None.

System Info

None.