ImportError: flash_attn not found when running grounded_sam2_florence2_image_demo.py

[XiaowenZhang-kuku]

Hello,

Thank you for your excellent work. When I try to run the following command:
python grounded_sam2_florence2_image_demo.py --pipeline object_detection_segmentation --image_path ./notebooks/images/cars.jpg
it gives me an error:
ImportError: This modeling file requires the following packages that were not found in your environment: flash_attn. Run pip install flash_attn

My environment is as follows::
CUDA=12.1
torch=2.3.1
transformers=4.33.2

Could you provide guidance on how to resolve this error?

[XiaowenZhang-kuku]

I tried to manually install flash-attention from GitHub releases, but I couldn't find a version that is compatible with my CUDA and PyTorch setup.
I installed several versions of flash-attention, but the error persisted: ImportError: This modeling file requires the following packages that were not found in your environment: flash_attn.

[rentainhe]

Have you tried pip install flash_attn as the error info says? I think it will automatically detect the environment you used and choose a suitable version for your environment.

[XiaowenZhang-kuku]

Have you tried pip install flash_attn as the error info says? I think it will automatically detect the environment you used and choose a suitable version for your environment.

Yes, this is my first attempt. However, error still exists.

[rentainhe]

Have you tried pip install flash_attn as the error info says? I think it will automatically detect the environment you used and choose a suitable version for your environment.

Yes, this is my first attempt. However, error still exists.

You mean you have successfully run pip install flash_attn but there're still the same package not found bug or you could not even install flash_attn

[XiaowenZhang-kuku]

have successfully run pip install flash_attn but there're still the same package not found

I have successfully run pip install flash_attn but there're still the same package not found.

[rentainhe]

Hi @XiaowenZhang-kuku , I think there are some issues have the same situation as you in flash_attn official repo, you can refer to this issue: Dao-AILab/flash-attention#453 to see if this can help you with your problem.