Add names of datasets

[tceron]

Dear mantainers,
I've been working with different LLM's pretraining data and I've noticed that even though the dataset has a unique name such as "dolmino-mix-1124", it encompasses other well-known datasets. For example, "dolmino-mix-1124" contains DCLM and FLAN. Since this information is relevant for people who are working with pre- and post-training data, I was wondering whether we could add a column for the datasets included in the training data of models and a way of comparing them across models given that there's a long list of datasets. Thank you!

[liesenf]

Good point. We try to stay on top of dataset use via the details in the yaml files.
See e.g. Apertus -> base model data which lists links to individual data releases
https://github.com/Language-Technology-Assessment/main-database/blob/preview/Apertus.yaml

But it has proven challenging to do this for more models because the relevant information is hard to find and indeed sometimes dataset are simply rebranded or remixed in obscure ways.

I think a more comprehensive investigation of how various datasets are used in different models would be a great addition but require considerable effort to investigate and quality control.

Perhaps you know more about dataset use in a specific model? Please consider adding this info to the relevant yaml files under base or enduser model data.