Skip to content

openai[minor]: add image generation to responses api #31424

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

eyurtsev
Copy link
Collaborator

@eyurtsev eyurtsev commented May 29, 2025

Does not support partial images during generation at the moment. Before doing that I'd like to figure out how to specify the aggregation logic without requiring changes in core.

Copy link

vercel bot commented May 29, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
langchain ⬜️ Ignored (Inspect) Visit Preview May 30, 2025 8:14pm

@eyurtsev eyurtsev requested a review from ccurme May 29, 2025 20:43
Copy link

codspeed-hq bot commented May 29, 2025

CodSpeed Walltime Performance Report

Merging #31424 will not alter performance

Comparing eugene/responses_api_2 (a832af7) with master (5bf8962)

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

Summary

✅ 13 untouched benchmarks

Copy link

codspeed-hq bot commented May 29, 2025

CodSpeed Instrumentation Performance Report

Merging #31424 will not alter performance

Comparing eugene/responses_api_2 (a832af7) with master (5bf8962)

Summary

✅ 13 untouched benchmarks

@eyurtsev eyurtsev self-assigned this May 30, 2025
@eyurtsev eyurtsev marked this pull request as ready for review May 30, 2025 18:19
@dosubot dosubot bot added the size:M This PR changes 30-99 lines, ignoring generated files. label May 30, 2025
@@ -118,6 +118,15 @@

_FUNCTION_CALL_IDS_MAP_KEY = "__openai_function_call_ids__"

WellKnownTools = (
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we import this from core

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not at the moment. We should do something in the reverse direction likely -- core should not know anything specific about the responses API. We should resolve in a separate refactor to figure out how to decouple core completely from openai specifics.


if tool["type"] == "image_generation" and "partial_images" in tool:
raise NotImplementedError(
"Partial image generation is not yet supported via the LangChain ChatOpenAI client. "
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mild preference for dropping partial_images and emitting a warning.

alternatively, could update the error message here to make it very obvious what to do (drop partial_images from the image_generation tool definition)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the doc-string

@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels May 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size:L This PR changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants