Parametrize FPS group (#9648)

* Parametrize FPS group

Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com>

* Apply isort and black reformatting

Signed-off-by: mikolajblaz <mikolajblaz@users.noreply.github.com>

* Change deafult to False

Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com>

* Add logic to new ckptIO

Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com>

* Turn on parallel save by default

Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com>

---------

Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com>
Signed-off-by: mikolajblaz <mikolajblaz@users.noreply.github.com>
Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com>

nemo/lightning/pytorch/strategies.py

@@ -148,6 +148,7 @@ def __init__(
         ckpt_torch_dist_multiproc=None,  ## TODO(ashors): put elsewhere?
         ckpt_assume_constant_structure=False,
         ckpt_parallel_save=True,
+        ckpt_parallel_save_within_dp=False,
         ckpt_parallel_load=False,
         ckpt_parallel_save_optim=True,
         setup_optimizers: bool = True,