Python: Add file handling support to BinaryContent for OpenAI Responses API

[ltwlf]

Summary

Enhances BinaryContent to support file handling for OpenAI Responses API, enabling file uploads through the responses agent while maintaining a provider-agnostic design.

Changes

BinaryContent Enhancements

• Add can_read property: Indicates whether content has readable data available
• Add from_file() class method: Creates BinaryContent instances from file paths with automatic base64 encoding
• Fix Unicode handling: Prevents decode errors when processing binary files (PDFs, images, etc.)

OpenAI Responses Agent Integration

• Add BinaryContent support: Pattern matching case for file handling in responses_agent_thread_actions.py
• Correct OpenAI API format: Uses proper filename and file_data structure with data URI format
• UUID-based filenames: Generates appropriate filenames with mime-type extensions
• Provider-specific mapping: File format conversion happens only in OpenAI agent code

Testing

• Test coverage: New tests for can_read, from_file(), and binary data handling
• Unicode error prevention: Specific test for binary PDF-like content
• Base64 encoding verification: Ensures proper data format for API compatibility

Design Principles

• Provider-agnostic: BinaryContent remains completely generic with no OpenAI-specific dependencies
• Clean separation: OpenAI format mapping isolated to OpenAI agent files
• No FileContent class: Enhances existing BinaryContent instead of introducing new types
• Follows existing patterns: Similar approach to ImageContent and TextContent handling

Usage Example

response = await self.agent.get_response(
      messages=ChatMessageContent(
          content="Analyse PDF",
          role=AuthorRole("user"),
          items=[BinaryContent.from_file(file_path=c":/test.pdf")],
      )
  )

[eavanvalkenburg]

Haven't looked yet, but why can't we use BinaryContent for this?

[ltwlf]

Binary is meant to be abstract (mentioned in code comments). ImageContent and FileContent derive from it. It also needed mapping to match open ai api schema spec for pdf file as base64. FileContent adapts the existing ImageContent implementation.

[moonbox3]

Hi @ltwlf, can you please check your Ruff settings? There are a lot of extra line adds and such in this PR, that aren't necessary -- most of the initial changes in I see responses_agent_thread_actions for example. It looks like you have some settings overriding our pyproject.toml Ruff settings that configure the line-length as 120?

The current code has the following following 99 chars (allowed because < 120):

from openai.types.responses.response_content_part_added_event import ResponseContentPartAddedEvent

and you're trying to commit the change that brings the first import line to 71 chars:

from openai.types.responses.response_content_part_added_event import (
    ResponseContentPartAddedEvent,
)

[ltwlf]

@moonbox3 thanks for the feedback and you are right about the unnecessary formatting changes.

I tracked down the issue - the root .vscode/settings.json was using autopep8 instead of ruff, which caused imports to be formatted differently than the project's pyproject.toml configuration.

I've fixed the VSCode settings and reverted all the unrelated formatting changes. The PR now only contains the minimal changes needed for FileContent functionality, and future contributions won't have this formatting issue.

[eavanvalkenburg]

@ltwlf we had a chat internally, we will eventually align the design of this with the dotnet Microsoft.Extensions.AI design, which only has BinaryContent without subclasses, so we would prefer to update BinaryContent instead of introducing something new that we then have to remove again, could you adapt this PR to do that (or create a new one)? Should be mostly adapting some of the docstrings in binarycontent and then using that instead of the filecontent for OpenAI Responses! Thanks for your efforts though, much appreciated!

[ltwlf]

@eavanvalkenburg thanks for the feedback! To clarify, are you suggesting that the binary type should handle OpenAI file data by default, with ImageContent inheriting from binary and overriding methods specifically for image handling?

[eavanvalkenburg]

@ltwlf exactly, eventually we might remove the ImageContent altogether, but that is breaking so we won't any time soon.

[eavanvalkenburg]

couple of small comments, overall it looks good. Do fix the linting settings because a lot of changes come from that which is annoying!

[markwallace-microsoft]

Python Test Coverage Report •

File	Stmts	Miss	Cover	Missing
agents/open_ai
responses_agent_thread_actions.py	364	126	65%	169, 186, 197, 208–209, 376–378, 392, 397, 404, 422–423, 427, 471–473, 476, 494–495, 504–506, 510, 514–515, 519–521, 532–534, 556, 558–559, 566–567, 569–570, 572, 574–575, 577–578, 661–663, 675–676, 678–682, 684–690, 692–701, 703–704, 706–709, 716–718, 723–724, 726–731, 733, 737–738, 740, 744, 748, 753–754, 759, 762–763, 768, 770, 829–832, 873–874, 878–880, 882, 907–908, 910–914, 916, 926–927, 935, 939, 946, 952, 1016
contents
binary_content.py	133	15	88%	99, 120, 128, 134, 147, 152–153, 204–205, 235–240
TOTAL	27052	4599	83%

Python Unit Test Overview

Tests	Skipped	Failures	Errors	Time
3624	18 💤	0 ❌	0 🔥	1m 50s ⏱️

[eavanvalkenburg]

Looks great, thanks for the effort!

[ltwlf]

@eavanvalkenburg now it is ready to review :)

[ltwlf]

I fixed some lint issues. The pre-commit hooks in my project weren’t working, so I reinstalled them.

[ltwlf]

@moonbox3 – I’ve added the examples you requested. Could you take another look when you have a moment? I’m a bit concerned we might run into merge conflicts if this PR stays open much longer. Thanks!