Add the Asynchronous Dataloader

[HenryJia]

Firstly, really sorry it's been this long. I've finally got round all my uni stuff and graduated and actually have time now.

Before submitting

• Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure to update the docs?
• Did you write any new necessary tests?

What does this PR do?

Add the asynchronous dataloader as discussed in

Lightning-AI/pytorch-lightning#1521
Lightning-AI/pytorch-lightning#1454
Lightning-AI/pytorch-lightning#1404
Lightning-AI/pytorch-lightning#1316

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Yes I had fun coding :)

[pep8speaks]

Hello @HenryJia! Thanks for updating this PR.

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2020-07-26 20:10:09 UTC

[codecov]

Codecov Report

Merging #127 into master will decrease coverage by 0.87%.
The diff coverage is 24.52%.

@@            Coverage Diff             @@
##           master     #127      +/-   ##
==========================================
- Coverage   92.25%   91.38%   -0.88%     
==========================================
  Files          79       80       +1     
  Lines        4031     4084      +53     
==========================================
+ Hits         3719     3732      +13     
- Misses        312      352      +40     

Flag	Coverage Δ
#unittests	91.38% <24.52%> (-0.88%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
pl_bolts/datamodules/async_dataloader.py	23.07% <23.07%> (ø)
pl_bolts/datamodules/__init__.py	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9ef90b4...5ca0f78. Read the comment docs.

[williamFalcon]

@HenryJia nice. Mind fixing the coverage?

[HenryJia]

@HenryJia nice. Mind fixing the coverage?

@williamFalcon Not entirely sure how to do that. The test requires a GPU to run since the dataloader needs a GPU (since to(device, non_blocking=True) requires device to be GPU), and I don't think any of the CI tests run on GPU machines?

[williamFalcon]

@Borda can we also add a set of GPUs for bolts?

Ok, we’ll merge now and make sure to add the gpus here as well
@edenlightning

[williamFalcon]

@HenryJia
i noticed the docs mention this works with datamodules. However, this is meant for dataloaders no?

[Borda]

@Borda can we also add a set of GPUs for bolts?
Ok, we’ll merge now and make sure to add the gpus here as well

yes, we were working on it with @luiscape as the best way is having Drone auto-scaling because these tests may run much longer the for PL...

[HenryJia]

@HenryJia
i noticed the docs mention this works with datamodules. However, this is meant for dataloaders no?

@williamFalcon
Yes it's for wrapping datasets and dataloaders. I meant it could be used inside a datamodule in its construction, should I edit that to make it clearer?

[williamFalcon]

yes. that’s not clear from the docs right now