Support OpenAI seed parameter

[simonw]

This should result in deterministic output: https://platform.openai.com/docs/api-reference/chat/create#completions/create-seed

This feature is in Beta. If specified, our system will make a best effort to sample deterministically, such that repeated requests with the same seed and parameters should return the same result. Determinism is not guaranteed, and you should refer to the system_fingerprint response parameter to monitor changes in the backend.

[simonw]

Prototype:

@@ -201,6 +201,9 @@ class Chat(Model):
             ),
             default=None,
         )
+        seed: Optional[int] = Field(
+            description="Integer seed to attempt to sample deterministically", default=None
+        )
 
         @field_validator("logit_bias")
         def validate_logit_bias(cls, logit_bias):

Result:

$ llm 'a joke about an otter, with a flourish' -o seed 1
Why did the confident otter never get lost?

Because he always had a "sea"cret maps-tache! With a flourish, he would whisk out his magnificent mustache, pointing to a hidden path through the waterways.
$ llm 'a joke about an otter, with a flourish' -o seed 1
Why did the confident otter never get lost?

Because he always had a "sea"cret maps-tache! With a flourish, he leads the way, swimming and whiskering his friends to new adventures!

Not quite the same though!

[simonw]

This worked:

$ llm -m 4t 'a joke about a badger and a clown' -o seed 1
Why did the badger get a job at the circus?

Because the clown said he needed someone who knew how to "badger" the audience into laughter!
$ llm -m 4t 'a joke about a badger and a clown' -o seed 1
Why did the badger get a job at the circus?

Because the clown said he needed someone who knew how to "badger" the audience into laughter!
$ llm -m 4t 'a joke about a badger and a clown' -o seed 3
Sure, here's a lighthearted joke for you:

Why did the badger get hired at the circus?

Because he was the only one who could make the clown stop laughing by just burrowing into the conversation!

[SovereignRemedy]

I have observed that except for special cases or when the answer consumes a small number of tokens, the seed fixed answer is rarely used. I wonder what is the test situation of your side