Devstral models rewrite all prefix code (probably a prompt issue)

[adoyxx]

Before submitting your bug report

• I've tried using the "Ask AI" feature on the Continue docs site to see if the docs have an answer
• I'm not able to find a related conversation on GitHub discussions that reports the same bug
• I'm not able to find an open issue that reports the same bug
• I've seen the troubleshooting guide on the Continue Docs

Relevant environment info

- OS: Linux Mint 22.2
- Continue version: 1.2.11
- IDE version: VS Codium 1.106
- Model: devstral-small-latest AND devstral-medium-latest
- config:
  
name: Local Agent
version: 1.0.0
schema: v1
models:
  - name: Codestral
    provider: mistral
    model: codestral-latest
    apiKey: API_KEY
    roles:
      - autocomplete
      - edit
      - chat
  - name: Devstral
    provider: mistral
    model: devstral-medium-latest
    apiKey: API_KEY
    roles:
      - edit
      - chat
  - name: Devstral Small
    provider: mistral
    model: devstral-small-latest
    apiKey: API_KEY
    roles:
      - edit
      - chat

Description

Issue

When editing code in Devstral and Devstral Small, the model rewrites not only the selected section but also the surrounding code. It should only modify the highlighted part while keeping the rest intact.

Workaround

I changed the prompt for these models to this:

Consider the following code:
Context before the editable section:
\`\`\`{{{language}}}
{{{prefix}}}
\`\`\`
Editable section (modify only this part):
\`\`\`{{{language}}}
{{{codeToEdit}}}
\`\`\`
Context after the editable section:
\`\`\`{{{language}}}
{{{suffix}}}
\`\`\`

Edit the code above to perfectly satisfy the following user request:
{{{userInput}}}

Output ONLY the modified editable section.
Do not rewrite the context before or after.
Leave existing comments in place unless changes require modifying them.
Do not include explanations, tags, or extra markdown.

It helped 😃

To reproduce

1. Highlite some lines of code and go to edit mode (Ctrl + I)

2. Choose Devstral model, type some prompt and send it

3. Result:

Log output

[sestinj]

Related issues:

• Code autocompletion with Qwen2.5 7B base and VLLM outputs garbage results. The prefix in the prompt_template is not being used. #3372 - Code autocompletion with Qwen2.5 7B base and VLLM outputs garbage results. The prefix in the prompt_template is not being used.
• Autocomplete is giving regular chat output alongside suggestion #6637 - Autocomplete is giving regular chat output alongside suggestion (Codestral/Devstral models)
• Autocomplete with codes uses https://codestral.mistral.ai/v1/chat/completions instead https://codestral.mistral.ai/v1/fim/completions #7178 - Autocomplete with codestral uses https://codestral.mistral.ai/v1/chat/completions instead https://codestral.mistral.ai/v1/fim/completions
• OpenAI model does not recognize assitant role and returns verbose completions #8897 - OpenAI model does not recognize assistant role and returns verbose completions