-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Description
Description
We're using this pattern (https://ai.pydantic.dev/output/#streaming-model-responses) to stream structured model responses and this pattern (https://ai.pydantic.dev/output/#output-validator-functions) to validate a JSON field foo we get back.
We pass allow_partial=not last to run partial validation on incomplete responses. This works great for the built-in validators the library creates. Unfortunately, in our custom output validator, we don't have access to the value of allow_partial.
This means our custom validator has to consider the response invalid the entire time it's streaming foo.
This is problematic is this case:
partial response 1: {"text": "This is an unfinished response that"
partial response 2: {"text": "This is an unfinished response that will remain unfinished to the user until foo finishes streaming", "foo": [{"type"
It would be great if the value of allow_partial passed to validate_response_output was accessible inside of custom output validators, so we could ignore errors if allow_partial is True.
References
No response