FusedRMSNormAffineMixedDtypesFunction is not importable in the PyTorch build without distributed support

[IvanYashchuk]

Describe the Bug
from apex.transformer import pipeline_parallel is not guarded by the torch.distributed.is_available() function resulting in import problems for non-distributed specific parts of Apex.

Minimal Steps/Code to Reproduce the Bug

1. Modify is_available() function to return False in torch/distributed/init.py
2. Modify is_available() function to return False in torch/distributed/rpc/init.py
3. Run

from apex.normalization.fused_layer_norm import FusedRMSNormAffineMixedDtypesFunction

Traceback:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.12/dist-packages/apex/__init__.py", line 27, in <module>
    from . import transformer
  File "/usr/local/lib/python3.12/dist-packages/apex/transformer/__init__.py", line 4, in <module>
    from apex.transformer import pipeline_parallel
  File "/usr/local/lib/python3.12/dist-packages/apex/transformer/pipeline_parallel/__init__.py", line 1, in <module>
    from apex.transformer.pipeline_parallel.schedules import get_forward_backward_func
  File "/usr/local/lib/python3.12/dist-packages/apex/transformer/pipeline_parallel/schedules/__init__.py", line 3, in <module>
    from apex.transformer.pipeline_parallel.schedules.fwd_bwd_no_pipelining import (
  File "/usr/local/lib/python3.12/dist-packages/apex/transformer/pipeline_parallel/schedules/fwd_bwd_no_pipelining.py", line 10, in <module>
    from apex.transformer.pipeline_parallel.schedules.common import Batch
  File "/usr/local/lib/python3.12/dist-packages/apex/transformer/pipeline_parallel/schedules/common.py", line 9, in <module>
    from apex.transformer.pipeline_parallel.p2p_communication import FutureTensor
  File "/usr/local/lib/python3.12/dist-packages/apex/transformer/pipeline_parallel/p2p_communication.py", line 25, in <module>
    from apex.transformer.utils import split_tensor_into_1d_equal_chunks
  File "/usr/local/lib/python3.12/dist-packages/apex/transformer/utils.py", line 11, in <module>
    torch.distributed.all_gather_into_tensor = torch.distributed._all_gather_base
                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'torch.distributed' has no attribute '_all_gather_base'

Expected Behavior

No import errors.