Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Ensure consistent language in coder prompt descriptions #1907

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

fry69
Copy link
Collaborator

@fry69 fry69 commented Oct 3, 2024

fix #1850

Thanks to @businistry for reporting and @jorgecolonconsulting for digging into this and providing the fix!

@fry69 fry69 mentioned this pull request Oct 3, 2024
@jorgecolonconsulting
Copy link

you're welcome!

@paul-gauthier
Copy link
Collaborator

Why do we think this rephrasing "works better" in general? It seems quite similar.

@fry69
Copy link
Collaborator Author

fry69 commented Oct 5, 2024

Why do we think this rephrasing "works better" in general? It seems quite similar.

Mystery of LLMs I guess? There is a slightly different meaning or instruction logic the new wording, I think.

@jorgecolonconsulting
Copy link

Why do we think this rephrasing "works better" in general? It seems quite similar.

Not so much “think” as in actually tested against a couple of different models. The biggest thing it fixes is where the default strong model (gpt-4o) that Aider uses responds in Spanish as many have reported here. That leaves a bit of a bad first impression when you’re expecting Aider to respond in your expected language.

LLMs are weird in that slight changes in a prompt can have dramatic changes in behavior for unknown reasons. Even if they are essentially saying the same thing. I’m sure you’ve experienced that yourself.

Have a look here where I was able to reproduce the problem and where this prompt did better: #1850 (comment)

@jorgecolonconsulting
Copy link

A thorough test across many models can make sure that prompt did better. We could even ask the issue reporters that experienced responses in Spanish what their chat history and incorporate those into the tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Answering in Spanish
3 participants