Token counting in Auto-compact uses provider metadata#3788
Merged
Conversation
DOsinga
approved these changes
Aug 3, 2025
| session_metadata: Option<&crate::session::storage::SessionMetadata>, | ||
| ) -> Result<CompactionCheckResult> { | ||
| // Get threshold from config or use override | ||
| let config = Config::global(); |
Collaborator
There was a problem hiding this comment.
let's kill the environment variables - this function should just have the threshold passed in. some caller, somewhere may use the environment variable for now. but not this deep into things
Collaborator
Author
There was a problem hiding this comment.
Where do we draw the line what is/isn't allowed to read the global config? I do agree with you that it would be more clear if this parameter got passed in but if there are too many params it leads to a lot of clutter.
| let context_limit = estimate_target_context_limit(provider); | ||
| let context_limit = provider.get_model_config().context_limit(); | ||
|
|
||
| // Try to use actual token counts from session metadata first |
| let (current_tokens, token_source) = match session_metadata.and_then(|m| m.total_tokens) { | ||
| Some(tokens) => (tokens as usize, "session metadata"), | ||
| None => { | ||
| // Fall back to estimated counts |
| }; | ||
|
|
||
| // Calculate usage ratio | ||
| let usage_ratio = current_tokens as f64 / context_limit as f64; |
Collaborator
There was a problem hiding this comment.
should we guard against division by zero here?
Collaborator
Author
There was a problem hiding this comment.
provider.get_model_config().context_limit() has a default value of 128k so can't get 0 div here.
| tokens_after: Some(tokens_after), | ||
| tokens_before: Some(tokens_before + SYSTEM_PROMPT_TOKEN_OVERHEAD + TOOLS_TOKEN_OVERHEAD), | ||
| tokens_after: Some(tokens_after + SYSTEM_PROMPT_TOKEN_OVERHEAD + TOOLS_TOKEN_OVERHEAD), | ||
| }) |
Collaborator
There was a problem hiding this comment.
This seems wrong; we know how many tokens we had before (sometimes estimated, but for most models we should get a result from them directly). So we also know what the overhead is; the actual tokens before - tokens_before as calculated here. then we can add that to tokens_after for the real value
Collaborator
Author
There was a problem hiding this comment.
This is because the summary prompt doesn't include the system prompt on either end.
So this estimate is pretty reasonable. #3721 adds the return values from the summarizer so will switch to those instead on there.
| @@ -530,4 +558,137 @@ mod tests { | |||
| .set_param("GOOSE_AUTO_COMPACT_THRESHOLD", serde_json::Value::from(0.3)) | |||
Collaborator
There was a problem hiding this comment.
this will create side effects when it fails
Collaborator
Author
There was a problem hiding this comment.
It gets unset at the end, but yes definitely can see why these can get leaky.
| let messages = vec![create_test_message("Hello"), create_test_message("World")]; | ||
|
|
||
| let result = check_compaction_needed(&agent, &messages, Some(0.3)) | ||
| let result = check_compaction_needed(&agent, &messages, Some(0.3), None) |
Collaborator
There was a problem hiding this comment.
do we ever test the meta_data in these calls? I'd maybe kill a few of these tests and have that does
…ng-quickfix * 'main' of github.com:block/goose: (26 commits) docs: Add YouTube link to Git MCP Tutorial (#3831) feat: more robust client initialization for the app (#3830) Build app bundles on release branches always (#3789) fix param order of debug_conversation_fixer (#3796) Fix directory switcher not working in active chat sessions and file browser not defaulting to current session directory path (#3791) File completion in CLI (#3822) docs: Dynamic linux install buttons (#3810) tests: Add missing `#[serial]` to two tests (#3816) Chore: apply more clippy rules to prevent from code complexity (#3813) chore(mcp): Add helpers to parse parameters (#2821) feat: enable docusaurus respectPrefersColorScheme (#3746) fix session resume in new window (#3800) Add settings field documentation to recipe guides (#3809) chore(deps): bump on-headers and compression in /documentation (#3532) fix(ui): refresh provider related issues (#3385) feat: Add comprehensive Linux build support (#3673) developer: Optimize text_editor_view a bit (#3781) Override session name generator for ollama provider (#3710) docs: fix markdown for cognee tutorial (#3801) chore: Upgrade node (#3756) ...
michaelneale
added a commit
that referenced
this pull request
Aug 4, 2025
* main: (34 commits) Token counting in Auto-compact uses provider metadata (#3788) docs: Add YouTube link to Git MCP Tutorial (#3831) feat: more robust client initialization for the app (#3830) Build app bundles on release branches always (#3789) fix param order of debug_conversation_fixer (#3796) Fix directory switcher not working in active chat sessions and file browser not defaulting to current session directory path (#3791) File completion in CLI (#3822) docs: Dynamic linux install buttons (#3810) tests: Add missing `#[serial]` to two tests (#3816) Chore: apply more clippy rules to prevent from code complexity (#3813) chore(mcp): Add helpers to parse parameters (#2821) feat: enable docusaurus respectPrefersColorScheme (#3746) fix session resume in new window (#3800) Add settings field documentation to recipe guides (#3809) chore(deps): bump on-headers and compression in /documentation (#3532) fix(ui): refresh provider related issues (#3385) feat: Add comprehensive Linux build support (#3673) developer: Optimize text_editor_view a bit (#3781) Override session name generator for ollama provider (#3710) docs: fix markdown for cognee tutorial (#3801) ...
michaelneale
added a commit
that referenced
this pull request
Aug 5, 2025
* main: Token counting in Auto-compact uses provider metadata (#3788) docs: Add YouTube link to Git MCP Tutorial (#3831) feat: more robust client initialization for the app (#3830) Build app bundles on release branches always (#3789) fix param order of debug_conversation_fixer (#3796) Fix directory switcher not working in active chat sessions and file browser not defaulting to current session directory path (#3791) File completion in CLI (#3822)
michaelneale
added a commit
that referenced
this pull request
Aug 5, 2025
* main: (56 commits) Token counting in Auto-compact uses provider metadata (#3788) docs: Add YouTube link to Git MCP Tutorial (#3831) feat: more robust client initialization for the app (#3830) Build app bundles on release branches always (#3789) fix param order of debug_conversation_fixer (#3796) Fix directory switcher not working in active chat sessions and file browser not defaulting to current session directory path (#3791) File completion in CLI (#3822) docs: Dynamic linux install buttons (#3810) tests: Add missing `#[serial]` to two tests (#3816) Chore: apply more clippy rules to prevent from code complexity (#3813) chore(mcp): Add helpers to parse parameters (#2821) feat: enable docusaurus respectPrefersColorScheme (#3746) fix session resume in new window (#3800) Add settings field documentation to recipe guides (#3809) chore(deps): bump on-headers and compression in /documentation (#3532) fix(ui): refresh provider related issues (#3385) feat: Add comprehensive Linux build support (#3673) developer: Optimize text_editor_view a bit (#3781) Override session name generator for ollama provider (#3710) docs: fix markdown for cognee tutorial (#3801) ...
kathawthorne
added a commit
to kathawthorne/goose
that referenced
this pull request
Aug 5, 2025
…-files * upstream/main: (150 commits) fix: replace glob/grep tool with shell (block#3834) docs: Add Youtube Link to dev.to tutorial (block#3869) Changed app settings configuration form to match settings panels (block#3829) Tell the user to hit compact (block#3851) Pin @mcp-ui/client in package.json (block#3860) blog for mcp-jupyter server (block#3059) docs: Adding dev.to Tutorial & Update CLI Component (block#3828) Detect client disconnects and cancel tool calls (block#3782) Suppress ansi with pipes (block#3775) Fix leaky env variable causing flaky test (block#3761) Update gemini error msg (block#3847) Generic retry and error parsing (block#3558) Clear the current line on ctrl-c in line with other tools (block#3764) chore: upgrade morph to use new model with instruction (block#3745) add CODEOWNERS file with /documentation owners (block#3840) Token counting in Auto-compact uses provider metadata (block#3788) docs: Add YouTube link to Git MCP Tutorial (block#3831) feat: more robust client initialization for the app (block#3830) Build app bundles on release branches always (block#3789) fix param order of debug_conversation_fixer (block#3796) ... # Conflicts: # crates/goose-mcp/src/developer/mod.rs
kathawthorne
added a commit
to kathawthorne/goose
that referenced
this pull request
Aug 5, 2025
…e-editable-displayable-title * upstream/main: (134 commits) fix: optimise reading large file content (block#3767) fix: replace glob/grep tool with shell (block#3834) docs: Add Youtube Link to dev.to tutorial (block#3869) Changed app settings configuration form to match settings panels (block#3829) Tell the user to hit compact (block#3851) Pin @mcp-ui/client in package.json (block#3860) blog for mcp-jupyter server (block#3059) docs: Adding dev.to Tutorial & Update CLI Component (block#3828) Detect client disconnects and cancel tool calls (block#3782) Suppress ansi with pipes (block#3775) Fix leaky env variable causing flaky test (block#3761) Update gemini error msg (block#3847) Generic retry and error parsing (block#3558) Clear the current line on ctrl-c in line with other tools (block#3764) chore: upgrade morph to use new model with instruction (block#3745) add CODEOWNERS file with /documentation owners (block#3840) Token counting in Auto-compact uses provider metadata (block#3788) docs: Add YouTube link to Git MCP Tutorial (block#3831) feat: more robust client initialization for the app (block#3830) Build app bundles on release branches always (block#3789) ...
michaelneale
added a commit
that referenced
this pull request
Aug 5, 2025
* main: (33 commits) fix: optimise reading large file content (#3767) fix: replace glob/grep tool with shell (#3834) docs: Add Youtube Link to dev.to tutorial (#3869) Changed app settings configuration form to match settings panels (#3829) Tell the user to hit compact (#3851) Pin @mcp-ui/client in package.json (#3860) blog for mcp-jupyter server (#3059) docs: Adding dev.to Tutorial & Update CLI Component (#3828) Detect client disconnects and cancel tool calls (#3782) Suppress ansi with pipes (#3775) Fix leaky env variable causing flaky test (#3761) Update gemini error msg (#3847) Generic retry and error parsing (#3558) Clear the current line on ctrl-c in line with other tools (#3764) chore: upgrade morph to use new model with instruction (#3745) add CODEOWNERS file with /documentation owners (#3840) Token counting in Auto-compact uses provider metadata (#3788) docs: Add YouTube link to Git MCP Tutorial (#3831) feat: more robust client initialization for the app (#3830) Build app bundles on release branches always (#3789) ...
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Still a slight discrepancy in the auto-compact completed message (a few hundred tokens) from estimating the system prompt, but the triggering logic is now from the metadata + aligned to the context window indicator.