auto_dataloader can auto set pin_memory if appropriate

[vfdev-5]

🚀 Feature

Idea is to automatically set pin_memory=True if idist.backend() is nccl or default device is cuda.

In the code we can omit this param instead of

ignite/examples/contrib/cifar10/main.py

Line 199 in 6b88eb8

pin_memory="cuda" in idist.device().type,

[InCogNiTo124]

Hey, me again :)

So there is only one line that should be changed? Did I understand that right?

[vfdev-5]

Hey @InCogNiTo124 :)

1. It is about adding some code on L91
   

   ignite/ignite/distributed/auto.py

   Lines 90 to 92 in 6b88eb8

   	)
   	if idist.has_xla_support and idist.backend() == idist_xla.XLA_TPU and kwargs.get("pin_memory", False):

check if kwargs contains pin_memory and if not set to True if idist.device() is cuda.
Probably, more than one line.

2. update examples

• cifar10

[InCogNiTo124]

Sounds doable. And what about docs or tests?

[vfdev-5]

Sounds doable. And what about docs or tests?

Just say here :

ignite/ignite/distributed/auto.py

Lines 26 to 28 in 6b88eb8

	- if no sampler provided by user, `torch DistributedSampler` is setup.
	- if a sampler is provided by user, it is wrapped by :class:`~ignite.distributed.auto.DistributedProxySampler`.

that we auto set pin_memory=True for cuda devices if user does not provide pin_memory arg.

For tests, we can check pin_memory attribute of DataLoader here :

ignite/tests/ignite/distributed/test_auto.py

Line 15 in 6b88eb8

def _test_auto_dataloader(ws, nproc, sampler_name=None, dl_type=DataLoader):

if provided instance is DataLoader (can be MpDeviceLoader for XLA)...