Add support for prompt caching directives in BedrockChatClient for MEAI #3930
Description
Describe the feature
The BedrockChatClient in the AWSSDK.Extensions.Bedrock.MEAI extension currently lacks the ability to pass cache control directives for prompts, which is essential for optimizing multi-turn conversations. This feature is already supported in the main Bedrock client via the cacheControl property in prompt configurations, as documented here: https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html
The proposed addition would allow users to include cache directives (e.g., ephemeral caching) within the AIContent blocks, by exposing or passing through AdditionalProperties to the underlying Bedrock API calls.
Use Case
I'm building a multi-turn chat application -- currently using the Bedrock SDK but I'd like to make it more generic using BedrockChatClient. I frequently encounter high latency and costs in repeated inference calls for similar prompts across conversation turns unless it's cached.
Other providers cache automatically, but Claude needs specific points set.
Proposed Solution
Extend the BedrockChatClient class to support passing cache control directives. This could be achieved by:
- Allowing users to set AdditionalProperties on AIContent blocks to include Bedrock's cacheControl field (e.g., { "cacheControl": { "mode": "ephemeral" } }).
- Ensuring these properties are serialized and passed through to the Converse API in Bedrock, aligning with the main client's behavior.
This would require minimal changes to the existing code in BedrockChatClient.cs, primarily in the message construction and serialization logic.
Other Information
No response
Acknowledgements
- I may be able to implement this feature request
- This feature might incur a breaking change
AWS .NET SDK and/or Package version used
AWSSDK.Extensions.Bedrock.MEAI 4.0.2
Targeted .NET Platform
.NET Framework 4.8, .NET 6
Operating System and version
Windows 11