Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow shuffle when auto-batching disabled in DataLoader #35761

Closed
huww98 opened this issue Mar 31, 2020 · 2 comments
Closed

Allow shuffle when auto-batching disabled in DataLoader #35761

huww98 opened this issue Mar 31, 2020 · 2 comments
Labels
enhancement Not as big of a feature, but technically not a bug. Should be easy to fix module: dataloader Related to torch.utils.data.DataLoader and Sampler triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@huww98
Copy link
Contributor

huww98 commented Mar 31, 2020

🚀 Feature

Allow DataLoader(batch_size=None, shuffle=True)

Motivation

My model handles one sample a time. So, I don't need auto-batching, but I want my dataset to be shuffled. However, when I use DataLoader(batch_size=None, shuffle=True), it raises batch_size=None option disables auto-batching and is mutually exclusive with shuffle, and drop_last.

But I can't see why they are mutually exclusive.

Pitch

Allow DataLoader(batch_size=None, shuffle=True).

Basically, just remove the shuffle check from this line:

if shuffle or drop_last:

Alternatives

I can:

  • Use batch_size=1 then remove the batching dimension afterwards
  • Use sampler=RandomSampler(dataset)

But these makes me feel like I'm fighting with DataLoader. Why I cannot just use shuffle=True?

Additional context

cc @ssnl

@ssnl ssnl added the module: dataloader Related to torch.utils.data.DataLoader and Sampler label Mar 31, 2020
@ezyang ezyang added enhancement Not as big of a feature, but technically not a bug. Should be easy to fix triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module labels Apr 1, 2020
@ydaiming
Copy link
Contributor

ydaiming commented Jun 3, 2020

Hi, there. If you haven't, I create a PR and work on it.

@ssnl
Copy link
Collaborator

ssnl commented Jun 3, 2020

@ydaiming Feel free to open a PR and tag me for review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Not as big of a feature, but technically not a bug. Should be easy to fix module: dataloader Related to torch.utils.data.DataLoader and Sampler triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
4 participants