Added lora support for phi

[umarbutler]

This pull request adds support for lora training microsoft/phi-1_5 by creating a new target modules mapping for models of the model_type phi.

[pacman100]

Thank you @umarbutler for adding the target modules for phi-1.5B! ✨

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[BenjaminBossan]

Thanks, the change looks good. One question before merging: Do we want to update the list of supported models in the README.md as well? That should make it more discoverable.

Also, out of curiosity, does PEFT also work with FusedDense from flash_attn?

[pacman100]

It should work with FusedDense given that it inherits from nn.Linear https://github.com/Dao-AILab/flash-attention/blob/ce3e7280f81869fda78578e91e79e8d11763a3e3/flash_attn/ops/fused_dense.py#L139. Am I missing something, @BenjaminBossan?

[BenjaminBossan]

It should work with FusedDense given that it inherits from nn.Linear

I see, thanks for the pointer. I agree it looks like it should just work.