ML info: split

[pierrot0]

Add COCO2014 as an example of what split definition could look like when split is defined at the fileSet level.

In the end I think we want to be able to define split at all levels: FileObject, FileSet, and RecordSet Field, as this can be transversal to other definitions. So once we agree on this, I can send an example where splits are defined as a RecordSet Field, and another example where splits are defined on the FileObject directly.

[github-actions]

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

[benjelloun]

I was thinking we would do something much simpler for splits: Specify it on RecordSet, as a simple enum field, sth like
"split" : "TEST". This requires the dataset creator to always define a RecordSet for each split, but I think that's useful anyway, since we want the data to be accessible via the library.

Do you see a strong reason to define splits at other levels?

[pierrot0]

I have added Oxford 102 flower category dataset as another example, this time where splits are defined on the recordSet level.

This is more complicated than I had first remembered: they used lookups to define a record split. Tentatively went with a possible syntax which will need to be fixed, but could help the conversation.

In both examples, we are still missing the semantic understanding of the splits, to account for different wordings of the same concept people use ("val", "validation", "train", "dev", ...).

As for defining a recordSet for every split, that would be a possibility, but might lead to duplications of sometimes already complex datasets, basically defining 3 times (possibly more sometimes) the same recordSets, where the only difference between those would be the split.

A possibility would maybe to define generic attribute values to File and FileSet, and have the recordSet be able to refer to those in one of its fields.

[pierrot0]

As per offline discussions, I'll update the PR to push the split definitions to the RecordSet level, possibly using filename and full_path_name or something similar to extract the needed information there.

[pierrot0]

Please take another look, thanks!
I have only kept COCO, I can re-introduce flowers in another PR.

[pierrot0]

Open question:
how do we distinguish between "#{csvfile.csv}/filename" the column and the actual filename?
Maybe we want a specific syntax to extract content based on file type? (CSV, JSON, etc.)

Also, I have used a non existing mlcommons.org/definitions, because I was not able to find a wikidata for individual data sets splits (train, or test, or validation).