fix deepspeed available detection

[fxmarty]

As per title, make the deepspeed available function more robust as https://github.com/huggingface/accelerate/blob/69e4c3c54da3201eda288b500d138761e7a5221c/src/accelerate/utils/imports.py#L72

Having tests/deepspeed & get_env that adds tests/ in the path then makes is_deepspeed_availebl() returns True although it should not, and in turn trainer.py tries to import DeepSpeedSchedulerWrapper that is not imported in accelerate as accelerate rightfully detects that DeepSpeed is not available.

This issue makes the test tests/extended/test_trainer_ext.py::TestTrainerExt::test_run_seq2seq_apex fail when APEX is installed but DeepSpeed is not.

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[ydshieh]

Could you show us how to see the following?

Having tests/deepspeed & get_env that adds tests/ in the path then makes is_deepspeed_availebl() returns True

[fxmarty]

@ydshieh Reproduction (maybe add relevant breakpoint() and print sys.path as well):

docker run --rm -it --gpus all nvcr.io/nvidia/pytorch:23.08-py3 /bin/bash
pip list | grep apex  # apex is here!
git clone https://github.com/huggingface/transformers.git
cd transformers
pip install -e .[dev-torch]
pytest tests/extended/test_trainer_ext.py::TestTrainerExt::test_run_seq2seq_apex -s -vvvvv

[ydshieh]

I tried to do it in a simple way

import os
# given by `get_env`
os.environ["PYTHONPATH"] = "/transformers/src:/transformers/tests:"
# shows `/transformers/src:/transformers/tests:`
print(os.environ["PYTHONPATH"])
from transformers.deepspeed import is_deepspeed_available
print(is_deepspeed_available())

and it prints False. Therefore, I am not very certain about the issue. Even with tests/, still False

[fxmarty]

@ydshieh You would need to use: PYTHONPATH=/path/to/transformers/tests python -c "from transformers.deepspeed import is_deepspeed_available; print(is_deepspeed_available())" to reproduce the issue.

I believe setting PYTHONPATH within a python script does not change sys.path (or somthing like this).

[ydshieh]

Thank you @fxmarty . Nice fix.

[ydshieh]

For the record, to reproduce (inside our docker env)

PYTHONPATH=/transformers/tests python3 -c "from transformers.deepspeed import is_deepspeed_available; print(is_deepspeed_available())"

[ArthurZucker]

Good call, grateful for the detail pr description 😄