Support torch dataloader without torch formatting

[lhoestq]

In #5084 we make the torch formatting consistent with the map-style datasets formatting: a torch formatted iterable dataset will yield torch tensors.

The previous behavior of the torch formatting for iterable dataset was simply to make the iterable dataset inherit from torch.utils.data.Dataset to make it work in a torch DataLoader. However ideally an unformatted dataset should also work with a DataLoader. To fix that, datasets.IterableDataset should inherit from torch.utils.data.IterableDataset.

Since we don't want to import torch on startup, I created this PR to dynamically make the datasets.IterableDataset class inherit form the torch one when a datasets.IterableDataset is instantiated and if PyTorch is available.

>>> from datasets import load_dataset
>>> ds = load_dataset("c4", "en", streaming=True, split="train")
>>> import torch.utils.data
>>> isinstance(ds, torch.utils.data.IterableDataset)
True
>>> dataloader = torch.utils.data.DataLoader(ds, batch_size=32, num_workers=4)
>>> for example in dataloader:
...:     ...

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[lhoestq]

Need some more time to fix the tests, especially with pickle

[mariosasko]

Looks good!

This is probably the least hacky we can get here :)

[polinaeterna]

I like the hack :)

I just left a fix in the docs (not related to this PR).

And I actually don't quite understand the idea - what's the motivation behind making only IterableDataset compatible with torch DataLoader without setting the format explicitly?

[lhoestq]

And I actually don't quite understand the idea - what's the motivation behind making only IterableDataset compatible with torch DataLoader without setting the format explicitly?

Setting the format to pytorch = set the output types of the dataset to be pytorch tensors. However sometimes your dataset is not made of tensors but you still want to be able to use a pytorch DataLoader

[mariosasko]

A bit more context.

The arrow-backed Dataset supports DataLoader(ds) (even if the format is not "torch"), and we want to be able to do the same with IterableDataset for consistency. However, this is when the PyTorch internals come into play - an iterable dataset needs to be an instance of torch.utils.data.IterableDataset due to this check (notice there is no check for the map-style version). Hence the explicit subclassing in this PR.

[lhoestq]

Exactly :) Btw I just took your comments into account @polinaeterna , so feel free to review again

[corbyrosset]

@lhoestq just checking, does this change still preserve the fix to the "data duplicate when setting num_works > 1 with streaming data" issue from before?

#3423

[lhoestq]

Yes :)