Skip to content

Conversation

jdaigle
Copy link
Contributor

@jdaigle jdaigle commented Oct 12, 2025

Description

Implements support for prompt caching using the AdditionalProperties dictionary of both ChatMessage and AIContent. When constructing both the System and Message blocks, any ChatMessage or AIContext can include in its AdditionalProperties a "CachePoint" key which is an instance of CachePointBlock.

Inserting CachePointBlock in the System block requires passing a non-empty ChatMessage where Role = ChatRole.System. We cannot include "CachePoint in the AdditionalProperties of the ChatOptions as this becomes serialized as AdditionalModelRequestFields.

Motivation and Context

Fixes #3930

Testing

No existing test fixtures for this project. But I manually tested each case.

Screenshots (if appropriate)

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist

  • My code follows the code style of this project
  • My change requires a change to the documentation
  • I have updated the documentation accordingly
  • I have read the README document
  • I have added tests to cover my changes
  • All new and existing tests passed

License

  • I confirm that this pull request can be released under the Apache 2 license

@jdaigle jdaigle changed the base branch from main to development October 12, 2025 01:41
@dscpinheiro dscpinheiro self-requested a review October 13, 2025 19:38
Copy link
Contributor

@dscpinheiro dscpinheiro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution! For now, can you remove the extensions/test/BedrockMEAITests/BedrockMEAITests.NetStandard.csproj file (and the changes to extensions/AWSSDK.Extensions.sln)?

Since it's a new project we'd have to update our internal build system to actually run it, and I don't want that to block the (straightforward) change from being released.

@jdaigle jdaigle requested a review from dscpinheiro October 13, 2025 20:24
@jdaigle
Copy link
Contributor Author

jdaigle commented Oct 13, 2025

@dscpinheiro I reverted those changes. Thanks for the feedback.

Copy link
Contributor

@dscpinheiro dscpinheiro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Internal build succeeded (DRY_RUN-830fe31e-5a00-4031-ba4c-673e961d9712).

@dscpinheiro dscpinheiro merged commit 3975b0b into aws:development Oct 14, 2025
@dscpinheiro
Copy link
Contributor

This will be included in today's release of the SDK.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for prompt caching directives in BedrockChatClient for MEAI

3 participants