Adding extra content to OpenAI LLM. Improving function call grouping. #4170
Conversation
|
|
|
I don't see extra_content from the ChoiceDelta: |
My testing shows it working OpenAI SDK's Pydantic models use extra='allow' 1, which preserves any additional fields from the JSON response. We access it via getattr(delta, "extra_content", None). This allows providers to include additional metadata like thought signatures. Context around google though signatures: https://ai.google.dev/gemini-api/docs/thought-signatures?authuser=1#openai |
| name: str | ||
| arguments: str | ||
| call_id: str | ||
| extra_content: dict[str, Any] | None = None |
There was a problem hiding this comment.
I would still name this extra IMO
| extra_content: dict[str, Any] | None = None | |
| extra: dict[str, Any] | None = None |
| role: ChatRole | None = None | ||
| content: str | None = None | ||
| tool_calls: list[FunctionToolCall] = Field(default_factory=list) | ||
| extra_content: dict[str, Any] | None = None |
There was a problem hiding this comment.
Same here
| extra_content: dict[str, Any] | None = None | |
| extra: dict[str, Any] | None = None |
- Introduced `extra_content` field in `FunctionToolCall`, `ChoiceDelta`, and `ChatMessage` models to accommodate provider-specific data (e.g., Google thought signatures). - Updated `LLMStream` and related methods to handle `extra_content` during tool calls and message processing. - Enhanced serialization in OpenAI provider format to include `extra_content` where applicable. - Improved grouping logic for function calls to utilize `group_id` when available for better parallel processing.
russellmartin-livekit
force-pushed
the
rm/adding-extra-content-to-openai-llm
branch
from
069fcf3 to
e46596f
Compare
No description provided.