RemoveAcceleratorConnector.parallel_device_ids and deprecate Trainer.data_parallel_device_ids

[DuYicong515]

Proposed refactor

Remove the property AcceleratorConnector.parallel_device_ids and migrate its usage
https://github.com/PyTorchLightning/pytorch-lightning/blob/550d3a640d1bfeb731a16b2996d3160f0f7eb071/pytorch_lightning/trainer/connectors/accelerator_connector.py#L830-L832

Deprecate the property Trainer.parallel_device_ids
https://github.com/PyTorchLightning/pytorch-lightning/blob/550d3a640d1bfeb731a16b2996d3160f0f7eb071/pytorch_lightning/trainer/trainer.py#L2043-L2047

Alternative

parallel_device_ids are meant to return GPU device indexes. You can use the logic below

if isinstance(Trainer.accelerator, GPUAccelerator):
  if isinstance(Trainer.strategy, ParallelStrategy):
    return [device.index for device in Trainer.strategy.parallel_devices]
  elif isinstance(Trainer.strategy, SingleDeviceStrategy):
    return [0]
return []

Motivation

Part of the follow ups in #11449 are to deprecate unused properties in accelerator_connector.
The accelerator properties are not meant to be public, migrated its internal usage. The only internal usage in logger_connector is planned to remove in v1.6.
https://github.com/PyTorchLightning/pytorch-lightning/blob/bc191af178474eccdd912059f728cb4db22dd0d8/pytorch_lightning/trainer/connectors/logger_connector/logger_connector.py#L226

The Trainer.data_parallel_device_ids were also deprecated in favor of getting the GPU device indexes from Trainer.strategy.parallel_devices directly. The only Internal usage is planned to remove in v1.6, other usages are in tests.

https://github.com/PyTorchLightning/pytorch-lightning/blob/bc191af178474eccdd912059f728cb4db22dd0d8/pytorch_lightning/callbacks/gpu_stats_monitor.py#L136

Pitch

https://github.com/PyTorchLightning/pytorch-lightning/blob/550d3a640d1bfeb731a16b2996d3160f0f7eb071/pytorch_lightning/trainer/trainer.py#L2043-L2047

@property
def data_parallel_device_ids(self) -> Optional[List[int]]:
    rank_zero_deprecation(
            "The `Trainer.data_parallel_device_ids` is deprecated in v1.6."
    )
    if not isinstance(self.accelerator, GPUAccelerator):
        return None
    if isinstance(self.strategy, ParallelStrategy):
        return [i for i in range(len(self.strategy.parallel_devices))]
    elif isinstance(self.strategy, SingleDeviceStrategy):
        return [0] 
    return None

https://github.com/PyTorchLightning/pytorch-lightning/blob/4c4b9d540f065c41ad288a78110cecbf45b96409/pytorch_lightning/trainer/connectors/logger_connector/logger_connector.py#L226

parallel_device_ids = []
if isinstance(self.trainer.accelerator, GPUAccelerator):
    if isinstance(self.trainer.strategy, ParallelStrategy):
        parallel_device_ids = [i for i in range(len(self.trainer.strategy.parallel_devices))]
    elif isinstance(self.strategy, SingleDeviceStrategy):
       parallel_device_ids = [0]
 if gpu_id in parallel_device_ids:

cc @justusschock @awaelchli @rohitgr7 @tchaton @akihironitta

[DuYicong515]

cc @four4fish