Feat: Let providers configure a fast model for summarization #4228

katzdave · 2025-08-20T18:00:44Z

No description provided.

crates/goose/src/providers/base.rs

crates/goose/src/providers/openai.rs

DOsinga · 2025-08-20T18:40:25Z

crates/goose/src/providers/openai.rs

+        let model = get_model(&json_response);
+        emit_debug_trace(model_config, &payload, &json_response, &usage);
+        Ok((message, ProviderUsage::new(model, usage)))
+    }


this was previously existing, but now that we pass in the model, no need to fish it out of the json response

Got rid of it, but confused by this one since we already had the model name from before the request?

crates/goose/src/providers/openai.rs

DOsinga · 2025-08-20T18:43:21Z

crates/goose/src/providers/openai.rs

+    ) -> Result<(Message, ProviderUsage), ProviderError> {
+        // Use the fast model (gpt-4o-mini) for fast completions
+        let mut fast_config = self.model.clone();
+        fast_config.model_name = OPEN_AI_FAST_MODEL.to_string();


make this a method on config - config.for_fast_model() should return a copy of itself with the model_name swapped out for the smart model if that is available

DOsinga · 2025-08-20T22:04:09Z

crates/goose/src/providers/anthropic.rs

+        config: CustomProviderConfig,
+    ) -> Result<Self> {
+        // Set the default fast model for Anthropic
+        model.fast_model = Some("claude-3-5-haiku-latest".to_string());


you don't want to do this - custom_config means it is a provider that speaks anthropic but isn't. they will not have this model

ack. Done + applied to openAI/google.

DOsinga · 2025-08-20T22:19:25Z

crates/goose/src/providers/anthropic.rs

-    pub fn from_env(model: ModelConfig) -> Result<Self> {
+    pub fn from_env(mut model: ModelConfig) -> Result<Self> {
+        // Set the default fast model for Anthropic
+        model.fast_model = Some("claude-3-5-haiku-latest".to_string());


make that into a constant on top - ideally we'd do this in a more elegant way, but let's do that later. don't make this mutable though, but have something like model = model.with_fast(CONSTANT) - possibly also check whether that is already set and don't overwrite it

done + applied to openAI/google

DOsinga · 2025-08-20T22:22:48Z

crates/goose/src/providers/base.rs

+    ///
+    /// # Errors
+    /// ProviderError
+    ///   - It's important to raise ContextLengthExceeded correctly since agent handles it


what are these comments? they repeat the existing comments which are now out of date demonstrating why we shouldn't have them in the first place

DOsinga · 2025-08-20T22:23:23Z

crates/goose/src/providers/base.rs

+    ///   - It's important to raise ContextLengthExceeded correctly since agent handles it
+    async fn complete_with_model(
+        &self,
+        model: &str,


this should be modelConfig, not model. if you pass a string but then you need model config, you're going to create one yourself

DOsinga · 2025-08-20T22:24:39Z

crates/goose/src/providers/bedrock.rs

-    async fn complete(
+    async fn complete_with_model(
        &self,
+        _model: &str,


now we are lying

* 'main' of github.com:block/goose: chore: upgrade rmcp to 0.6.0 (#4243) doc: uvx not npx (#4240) Add PKCE support for Tetrate Agent Router Service (#4165) Read AGENTS.md by default (#4232) docs: configure provider and model (#4235) docs: add figma tutorial (#4231) Add Nix flake for reproducible builds (#4213) Enhanced onboarding page visual design (#4156) feat: adds mtls to all providers (#2794) (#2799) Don't show a confirm dialog for quitting (#4225) Fix: Missing smart_approve in CLI /mode help text and error message (#4132)

* main: (108 commits) Remove unused game (#4226) fix issue where app redirects to home after initialization but user has already started a chat (#4260) Feat: Let providers configure a fast model for summarization (#4228) docs: update tool selection strategy (#4258) feat: upgrade `@mcp-ui/client` package and improve UI message handling (#4164) stop replacing chat window when changing working directory (#4200) Only fetch session tokens when chat state is idle to avoid resetting during streaming (#4104) bump timeouts for e2e tests (#4251) docs: custom context files improvements (#4096) chore: upgrade rmcp to 0.6.0 (#4243) doc: uvx not npx (#4240) Add PKCE support for Tetrate Agent Router Service (#4165) Read AGENTS.md by default (#4232) docs: configure provider and model (#4235) docs: add figma tutorial (#4231) Add Nix flake for reproducible builds (#4213) Enhanced onboarding page visual design (#4156) feat: adds mtls to all providers (#2794) (#2799) Don't show a confirm dialog for quitting (#4225) Fix: Missing smart_approve in CLI /mode help text and error message (#4132) ...

* main: docs: update View/Edit Recipe menu item name (#4267) Remove unused game (#4226) fix issue where app redirects to home after initialization but user has already started a chat (#4260) Feat: Let providers configure a fast model for summarization (#4228) docs: update tool selection strategy (#4258) feat: upgrade `@mcp-ui/client` package and improve UI message handling (#4164) stop replacing chat window when changing working directory (#4200) Only fetch session tokens when chat state is idle to avoid resetting during streaming (#4104) bump timeouts for e2e tests (#4251) docs: custom context files improvements (#4096) chore: upgrade rmcp to 0.6.0 (#4243) doc: uvx not npx (#4240) Add PKCE support for Tetrate Agent Router Service (#4165) Read AGENTS.md by default (#4232) docs: configure provider and model (#4235)

* main: (42 commits) feat: Add message queue system with interruption handling (#4179) Start extensions concurrently (#4234) Add X-Title and referer headers on exchange to tetrate (#4250) docs: update View/Edit Recipe menu item name (#4267) Remove unused game (#4226) fix issue where app redirects to home after initialization but user has already started a chat (#4260) Feat: Let providers configure a fast model for summarization (#4228) docs: update tool selection strategy (#4258) feat: upgrade `@mcp-ui/client` package and improve UI message handling (#4164) stop replacing chat window when changing working directory (#4200) Only fetch session tokens when chat state is idle to avoid resetting during streaming (#4104) bump timeouts for e2e tests (#4251) docs: custom context files improvements (#4096) chore: upgrade rmcp to 0.6.0 (#4243) doc: uvx not npx (#4240) Add PKCE support for Tetrate Agent Router Service (#4165) Read AGENTS.md by default (#4232) docs: configure provider and model (#4235) docs: add figma tutorial (#4231) Add Nix flake for reproducible builds (#4213) ...

) Signed-off-by: Alex Rosenzweig <arosenzweig@squareup.com>

) Signed-off-by: Dorien Koelemeijer <dkoelemeijer@squareup.com>

katzdave added 2 commits August 20, 2025 13:48

my hints for goose

e501e3b

base impl

060bbdd

Kvadratni reviewed Aug 20, 2025

View reviewed changes

crates/goose/src/providers/base.rs Outdated Show resolved Hide resolved

DOsinga approved these changes Aug 20, 2025

View reviewed changes

katzdave added 10 commits August 20, 2025 15:28

update new providers with api

8b34e02

add provider defaults

966e391

cleanup

f95b13d

clean comments

32256b0

Fmt

e8cd8fc

parse context limit

3c3c376

more model config

bdb5583

rm groq model

352e0c5

Reset python files

ee5f40b

should build now

0c85d4d

DOsinga reviewed Aug 20, 2025

View reviewed changes

katzdave added 14 commits August 21, 2025 00:00

with fast abstraction + env set

3637a1a

no fast model on custom config

5528a29

fn comments

0447587

Swap model to modelconfig

51bfa48

rm extra scripts

d3a5307

openai output model

55aeedf

fix warnings

b666551

fmt

f6c2550

support databricks

8cc66f1

bring back output

4ec0d9b

fix databricks

bdbfcf4

databricks to 3.7sonnet

7a5f390

fix clippy

1e6f164

summary model -> 1.5flash

395f434

katzdave changed the title ~~WIP: Let providers configure a fast model for summarization~~ Feat: Let providers configure a fast model for summarization Aug 21, 2025

katzdave added 6 commits August 21, 2025 11:26

fix titrate

f996262

one more test fix

9e43de9

fix agent tests

2f232f8

add fast model exists check

0cd4189

bump sonnet model

74a8359

katzdave merged commit 72f4ebc into main Aug 21, 2025
10 checks passed

katzdave deleted the dkatz/fast-summarize2 branch August 21, 2025 21:41

alexhancock mentioned this pull request Aug 22, 2025

release/1.6.0 #4280

Merged

shellz-n-stuff pushed a commit to shellz-n-stuff/goose that referenced this pull request Aug 27, 2025

Feat: Let providers configure a fast model for summarization (block#4228

b1122fc

) Signed-off-by: Alex Rosenzweig <arosenzweig@squareup.com>

dorien-koelemeijer pushed a commit to dorien-koelemeijer/goose that referenced this pull request Sep 2, 2025

Feat: Let providers configure a fast model for summarization (block#4228

aad7128

) Signed-off-by: Dorien Koelemeijer <dkoelemeijer@squareup.com>

Feat: Let providers configure a fast model for summarization #4228

Feat: Let providers configure a fast model for summarization #4228

Uh oh!

Conversation

katzdave commented Aug 20, 2025

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants