Skip to content

Pass allow_partial to custom output validators #3194

@petersli

Description

@petersli

Description

We're using this pattern (https://ai.pydantic.dev/output/#streaming-model-responses) to stream structured model responses and this pattern (https://ai.pydantic.dev/output/#output-validator-functions) to validate a JSON field foo we get back.

We pass allow_partial=not last to run partial validation on incomplete responses. This works great for the built-in validators the library creates. Unfortunately, in our custom output validator, we don't have access to the value of allow_partial.

This means our custom validator has to consider the response invalid the entire time it's streaming foo.
This is problematic is this case:

partial response 1: {"text": "This is an unfinished response that"
partial response 2: {"text": "This is an unfinished response that will remain unfinished to the user until foo finishes streaming", "foo": [{"type"

It would be great if the value of allow_partial passed to validate_response_output was accessible inside of custom output validators, so we could ignore errors if allow_partial is True.

References

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions