Skip to content

Commit 6fb8ab7

Browse files
Increase output token limit for EquivalenceEvaluator (#6835)
EquivalenceEvaluator was specifying MaxOutputTokens = 1 since its prompt instructs the LLM to produce a response (score) that is a single digit (between 1 and 5). Turns out that while this works for most models (including the OpenAI models that were used to test the prompt), some models require more than one token for this. For example, looks like Claude requires two tokens for this - see #6814). This PR bumps the MaxOutputTokens to 5 to address the above issue. Fixes #6814
1 parent 6e61568 commit 6fb8ab7

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

src/Libraries/Microsoft.Extensions.AI.Evaluation.Quality/EquivalenceEvaluator.cs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ public sealed class EquivalenceEvaluator : IEvaluator
5252
new ChatOptions
5353
{
5454
Temperature = 0.0f,
55-
MaxOutputTokens = 1,
55+
MaxOutputTokens = 5, // See https://github.com/dotnet/extensions/issues/6814.
5656
TopP = 1.0f,
5757
PresencePenalty = 0.0f,
5858
FrequencyPenalty = 0.0f,

0 commit comments

Comments
 (0)