feat: enable prompt cache for anthropic

[yingjiehe-xyz]

Enable prompt cache for Anthropic following https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching#how-prompt-caching-works.

Generally, we add "cache_control": {"type": "ephemeral"} into tool, system and message sections. The cache hit can be verified with the usage response, like

usage: {
cache_creation_input_tokens: 1479
cache_read_input_tokens: 0
input_tokens: 4
output_tokens: 78
}

And currently, “ephemeral” is the only supported cache type, which corresponds to this 5-minute lifetime.

Cost saving: From the pricing,

1. Cache write tokens are 25% more expensive than base input tokens
2. Cache read tokens are 90% cheaper than base input tokens

Assume we have N turns conversion, S denotes to system prompt length and the average new tokens(user inputs + outputs) length for each turn is M
Before cache: our estimated cost is around

1. S + (S + M) + (S + M * 2) + ... + (S + M * (N - 1)) = S * N + (N - 1) * N / 2 * M
   After cache: our estimated cost is around
2. (S + M * (N - 1)) * 1.25 + (S + (S + M) + ... + (S + M * (N - 1))) * 0.1 = (S + M * (N - 1)) * 1.25 + (S * N + (N - 1) * N / 2 * M) * 0.1
   To compare result 1 and result 2, we need to compare (S + M * (N - 1)) * 1.25 and (S * N + (N - 1) * N / 2 * M) * 0.9, if (S + M * (N - 1)) * 1.25 is greater, then cache is cost more and vice visa. Normally, our S is greater than 1000 and M is large as well, (S + M * (N - 1)) * 1.25 should be much smaller which means cache reduces our cost.
   And some estimation from anthropic post:
   

Test with just run-ui and response verification:

[github-actions]

Desktop App for this PR

The following build is available for testing:

• 📱 macOS Desktop App (Universal, signed)

The app is signed and notarized for macOS. After downloading, unzip the file and drag the Goose.app to your Applications folder.

This link is provided by nightly.link and will work even if you're not logged into GitHub.

[ahau-square]

Can you do a quick back of the envelope estimation of cost and savings for turning on prompt caching based on the Anthropic pricing?

[yingjiehe-xyz]

Can you do a quick back of the envelope estimation of cost and savings for turning on prompt caching based on the Anthropic pricing?

Sure, add one screenshot from anthropic post, also add my estimations

[ahau-square]

Can you do a quick back of the envelope estimation of cost and savings for turning on prompt caching based on the Anthropic pricing?

Sure, add one screenshot from anthropic post, also add my estimations

Awesome, looks like we can expect some big savings here!

[michaelneale]

this is really cool @yingjiehe-xyz and I think people will appreciate this a lot.

I wonder if similar exists for openrouter (as people like to use anthropic that way) but yes! very nice!

[yingjiehe-xyz]

this is really cool @yingjiehe-xyz and I think people will appreciate this a lot.

I wonder if similar exists for openrouter (as people like to use anthropic that way) but yes! very nice!

Yes, it is available in openrouter: https://openrouter.ai/docs/prompt-caching, I am planing on this for the next step