Expand AI Code Completion beyond Copilot and Supermaven

[zerocorebeta]

Check for existing issues

• Completed

Describe the feature

After going through: https://zed.dev/docs/completions

Zed currently supports completions via external LLM APIs like GitHub Copilot and Supermaven, but this is restrictive. Many users, for privacy or performance reasons, might prefer alternatives like Gemini Flash or local models via Ollama.

There are several advanced LLMs that support the Fill-in-the-Middle (FIM) objective, such as CodeGemma. Additionally, platforms like Continue.dev allow code completion via local models, with Startcoder via Ollama as the default.

Expanding LLM support to include more flexible, local, or privacy-focused options would greatly enhance Zed's appeal and utility for a wider range of developers.

If applicable, add mockups / screenshots to help present your vision of the feature

No response

[ggerganov]

We have recently extended the llama.cpp server with a specialized /infill endpoint that enables FIM requests with large contexts to run efficiently in local environments. A simple example of using this endpoint with Qwen2.5-Coder in Neovim is demonstrated here: ggerganov/llama.cpp#9787

I believe it could be an interesting option to explore in the scope of this issue. Feel free to ping me if you have any questions.

[20manas]

I think Zed AI should also provide its own code completion functionality.

[bersace]

Does Zed allows to use Codestral cloud for FIM ?