是否支持启动的时候，指定use_flash_attion_2 ??

[awzhgw]

是否支持启动的时候，指定use_flash_attion_2 ??

[LinB203]

支持。参考以下代码。除了Qwen以外都可以这样，因为Qwen会自动开启。
[En] It supports, just modify the code as follows except the Qwen-based, because it will auto-enable the flash-attn.

model = LlavaPhiForCausalLM.from_pretrained(
                    model_args.model_name_or_path,
                    cache_dir=training_args.cache_dir,
                    attn_implementation="flash_attention_2",  # add this line
                    **bnb_model_from_pretrained_args
                )

[LinB203]

Hi, we have tested the flash attention2, but we found the performance degradation. We found the same question....
Therefore, we do not recommend you guys to enable the flash attention2.
huggingface/transformers#28488