Add support for GPT-5 Free-Form Function Calling and Context Free Grammar constraints over tools #2572
Conversation
the response from the model is not correctly handled yet
now produces appropriate warnings when misconfigured
I can now call the model with the following:
```
from pydantic_ai import Agent
import asyncio
import json
from pydantic_core import to_jsonable_python
from pydantic_ai.models.openai import OpenAIResponsesModel
agent = Agent(OpenAIResponsesModel("gpt-5-mini"), model_settings={"openai_reasoning_effort": "minimal"})
@agent.tool_plain(free_form=True)
def execute_lucene_query(query: str) -> str:
"""Use this to run a lucene query against the system.
YOU MUST ALWAYS RUN A QUERY BEFORE ANSWERING THE USER.
Args:
query: the lucene query to run
Returns:
the result of executing the query, or an error message
"""
return "The query failed to execute, the solr server is unavailable"
async def run() -> None:
response = await agent.run("Execute the lucene query text:IKEA and give me the results")
history = response.all_messages()
as_json = json.dumps(to_jsonable_python(history), indent=2)
print(as_json)
print(response.output)
asyncio.run(run())
```
|
I'm going to work on CI checks after getting something that at least supports the CFG tool calls. |
will also validate the grammar if the dependency is installed
| strict: bool | None = None, | ||
| free_form: bool | None = None, | ||
| grammar_syntax: Literal['regex', 'lark'] | None = None, | ||
| grammar_definition: str | None = None, |
There was a problem hiding this comment.
@matthewfranglen I know you still have some work planned on this PR before it's really ready for review, but please consider the API I proposed in #2513 (comment). I'd prefer one argument taking an object over 3 that need to be used together
There was a problem hiding this comment.
Sorry I missed your intent with that part. It's a very good idea I will certainly do that, and it will clean up what I have done so far. Thanks for the reminder.
There was a problem hiding this comment.
I believe I have addressed this now
going to add the literal version now
it's provided by the init
pyright isn't happy with it
|
just coverage and documentation now, pending the actual review of course |
|
@matthewfranglen Just a heads-up that I'll be out this coming week and will be back the 25th. Assuming this is not urgent I'll review it then. If it is, please ping |
DouweM
changed the title
matthewfranglen
force-pushed
the
freeform-and-cfg-tools
branch
from
29d8030 to
febe88d
Compare
`tool_argument_name` variable exists to narrow type of `argument_name` to `str`
This separates the implementation of regex and lark to two classes. The base class of these would be empty, so the `TextFormat` has become a union type. The openai tool call handling has been changed to silently ignore formats that it does not handle. This is consistent with how the gpt-5 models ignore temperature parameter, which is not supported by reasoning models (see pydantic#2483).
|
@matthewfranglen I was thinking yesterday that another, more Pydantic-like, API for this this would be to annotate the The user could then choose to further annotate that type with a Pydantic validator so that we get server-side validation in addition to (if the model supports it) client-side validation. We could also expose It'd be cool if we could automatically use CFGs for any tool function taking a single string argument annotated with |
|
This PR is stale, and will be closed in 3 days if no reply is received. |
|
This PR is stale, and will be closed in 3 days if no reply is received. |
|
This PR is stale, and will be closed in 3 days if no reply is received. |
2 participants
This is the PR to implement free-form function calling and context free constraints over tools. This implements:
I'm hoping to get a working implementation done today, after which we can have the rounds of feedback and improvement required before this can be merged. I will try to keep in mind the feedback that I got on the last PR.
Since there is a specific company voice that you wish to maintain, when suggesting changes to documentation et al please provide the exact wording that you desire.