[Feature]: Will future versions of vllm support FlashMLA, the inference acceleration technology open-sourced by DeepSeek?

[Leander-wang]

🚀 The feature, motivation and pitch

I propose adding support for FlashMLA in future versions of vllm. FlashMLA is an inference acceleration technology developed by DeepSeek, designed to optimize large-scale transformer models by improving efficiency in both memory usage and computation time, especially during inference tasks.

related link: https://github.com/deepseek-ai/FlashMLA

Alternatives

No response

Additional context

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[bhaktatejas922]

would be great to see how it compares to PagedAttention

[gaocegege]

Had a few discussions about it in the slack https://vllm-dev.slack.com/archives/C08AD2B5HH8/p1740361881324199

Please join if you are interested.

[Leander-wang]

@gaocegege Thanks, I noticed that a PR has already been submitted: #13747. Can I understand that vllm will support this in the next version, or possibly the version after that?

[gaocegege]

From what I gathered in the Slack discussions, I think that’s the case.

[gaocegege]

By the way, since this issue is a duplicate of #13735, could we go ahead and close this one? We can continue tracking the discussion there

[Leander-wang]

@gaocegege Alright, as you wish.