Github Marketplace Provider: Maximum Context Legnth exceeded

[richtong]

OK, with the builtin GitHub Marketplace provider, I tried to use Phi4, and it looks like the requested context length exceeds the maximum.

Is there are way to reduce the context request for that provider? Not obvious from the documentation:

Failed to get response: 400
{"error":{"code":"Bad Request","message":"{\"object\":\"error\",\"message\":\"This model's maximum context length is 16384 tokens. However, you requested 20557 tokens (4173 in the messages, 16384 in the completion). Please reduce the length of the messages or completion.\",\"type\":\"BadRequestError\",\"param\":null,\"code\":400}","status":400}}

[richtong]

It could be that this has to do with rate limits I guess, when I look at https://docs.github.com/en/github-models/prototyping-with-ai-models and I'm guessing you need to be with a paid GitHub Copilot to get more access to this, but is there a way to change context length requests from the plugin?

There seem to be a bunch of limits, but I don't see where context length can be set and I don't see it in the default providers section in

CopilotChat.nvim/lua/CopilotChat/config/providers.lua

Line 288 in 23162ad

M.github_models = {

[deathbeam]

Nothing to do with rate limits, just limits of the model. Also Github Models are separate from Copilot, they are mostly just for testing, but the actual rate limits on them are very annoying. And you would need azure key and not copilot subscription afaik to increase the rate limits.

But anyway, the plugin was using incorrect value when specifying max output tokens when using github models so I fixed that so hopefully this should not work properly

[richtong]

Oh that's great, thanks @deathbeam which by the way is an awesome github name. We have an Azure key, so I'll add, I need to look at the code to see what Environment variable it uses to pick that up. I can also add a PR to explain the doc. Thanks!