Feature Request: Pixtral by Mistral support (pixtral-12b-240910)

[arch-btw]

Prerequisites

• I am running the latest code. Mention the version if possible as well.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

Dear llama.cpp team,

Mistral has just released Pixtral and I would like to request support for it, if possible.

Here are some relevant links:

X announcement: https://x.com/mistralai/status/1833758285167722836

Magnet link: xt=urn:btih:7278e625de2b1da598b23954c13933047126238a&dn=pixtral-12b-240910&tr=udp%3A%2F%http://2ftracker.opentrackr.org/%3A1337%2Fannounce&tr=udp%3A%2F%http://2fopen.demonii.com/%3A1337%2Fannounce&tr=http%3A%2F%http://2ftracker.ipv6tracker.org/%3A80%2Fannounce

HuggingFace alternative download link: https://huggingface.co/mistral-community/pixtral-12b-240910

Additional information: https://github.com/mistralai/mistral-common/releases/tag/v1.4.0

Important notes from the readme:

• Use GELU for the vision adapter
• Use 2D ROPE for the vision encoder

---

Thank you for your time and consideration!

[chigkim]

+1

[razorback16]

+1

[yukiarimo]

Does it work right now?

[Maplerxyz]

+1

[YorkieDev]

huggingface/transformers#33449 Support was just merged into huggingface transformers. Not sure if that makes it any easier to add support to llama.cpp

[itodorovic]

+1
Pixtral seem to be awesome, just tried it on https://chat.mistral.ai/chat/ and it works wonders. Would be really nice to use it with llama.cpp

[yukiarimo]

Any updates?

[btaskel]

+1

[cbrousseauAumni]

+1

[napa3um]

+1

[IngLP]

+1!

[Tianoklein]

Please!
Allow us to compare 2 or more images via API

[Malte0621]

+1