#7858 breaks IterableDataset with __len__ in Trainer

[cccntu]

transformers/src/transformers/trainer.py

Lines 381 to 382 in 08f534d

	if not isinstance(self.train_dataset, collections.abc.Sized):
	return None

This used to be (before #7858)

if isinstance(self.train_dataset, torch.utils.data.IterableDataset):

I am using IterableDataset with len in Trainer. This change makes it return a sampler and results in an error later. ValueError: DataLoader with IterableDataset: expected unspecified sampler option, but got sampler=<torch.utils.data.sampler.RandomSampler object at 0x7fa32c57b340>

Maybe change to this?

if (isinstance(self.train_dataset, torch.utils.data.IterableDataset) or
        not isinstance(self.train_dataset, collections.abc.Sized)):

@j-rossi-nl @sgugger

[sgugger]

I'm sorry, but you could explain to me why an IterableDataset with a __len__ is not a regular Dataset?

[cccntu]

In my case, I wrap a Dataset using a class that inherits IterableDataset, and defines a __len__().
The purpose is to implement smart batching[1]. I use IterableDataset so I can control how to iterate the data.

I don't know if it's possible/easier to Dataset+Sampler, if so please let me know.
Also note that (after the change) if I drop __len__() to suppress the bug, I would then need to specify max_iter (or something like that), which is inconvenient.
[1] (https://wandb.ai/pommedeterresautee/speed_training/reports/Train-HuggingFace-Models-Twice-As-Fast--VmlldzoxMDgzOTI)

[sgugger]

It does seem a bit hacky but I guess we can add that test. Do you want to suggest a PR with the change?