Add Tensor Parallel support for ALL models

[ArthurZucker]

Just opening this to add support for all models following #34184

Lets bring support to all model! 🤗

• Llama

It would be great to add the support for more architectures such as

• Qwen2
• QwenVl
• Mistral
• Llava

... and many more

For anyone who wants to contribute just open a PR and link it to this issue, and ping me for a review!! 🤗

[farrosalferro]

If it's okay, I want to take the Gemma and Gemma2 @ArthurZucker

[kmehant]

@ArthurZucker I will add support for Granite. Thanks

[VladOS95-cyber]

Hey @ArthurZucker! I am going to work on Mistral.

[VladOS95-cyber]

Hey @ArthurZucker! I am going to start working on Qwen2