Unable to use Devstral Small: error due to max tokens

[KTibow]

Some models (Devstral Small on Chutes, Mercury Coder, many more near the bottom of this list) have context lengths of 32k tokens. Since opencode always sets max tokens to 32k and sends more tokens as context, errors result:

This is likely due to an incorrect context length in models.dev. opencode should adapt its request to use a shorter max tokens (in case the data is incorrect) and/or switch to using data from the models endpoint (always use correct data).

[thdxr]

can you set output tokens number to be 16K or something smaller?

[KTibow]

That would theoretically fix it, but ideally anomalyco/models.dev#49 would be merged and then something like this would be added:

- const outputLimit = Math.min(model.info.limit.output, OUTPUT_TOKEN_MAX) || OUTPUT_TOKEN_MAX
+ let outputLimit = OUTPUT_TOKEN_MAX;
+ if (model.info.limit.output && model.info.limit.output < outputLimit) outputLimit = model.info.limit.output;
+ const tokens = previous && previous.tokens && (previous.tokens.input + previous.tokens.cache.read + previous.tokens.cache.write + previous.tokens.output);
+ const tokensRemaining = model.info.limit.input - tokens;
+ if (tokensRemaining && tokensRemaining < outputLimit) outputLimit = tokensRemaining;