dynamic subdirectory hint loading by johnmatthewtennant · Pull Request #5759 · block/goose

johnmatthewtennant · 2025-11-16T17:49:08Z

Summary

Goose reads the hints in the working directory but does not read hints in subdirectories. This is problematic for working in monorepos.

E.g. If you run Goose from the repo-root folder, then Goose reads repo-root/agents.md on startup.
If you direct goose to make changes to the frontend in repo-root/frontend, Goose will start reading and editing files without reading repo-root/frontend/agents.md.

This PR adds logic to dynamically load hints in subdirectories when Goose reads files in those subdirectories.
It also maintains a turn counter and prunes the hints from the context when Goose hasn't accessed files in the subdirectory in the last N turns.

I'm new to the codebase and leaned heavily on AI to produce the draft PR so I'd like some help assessing if this is the right approach.

Type of Change

AI Assistance

This PR was created or reviewed with AI assistance

Testing

Added unit & integration tests
Tested that hints in subdirectories load and are pruned in manual goose sessions

Related Issues

Relates to #5840

- Use existing turns_taken counter instead of separate turn_counter - Rename 'context' to 'hints' throughout (more consistent terminology) - Rename load_agents_from_directory to load_hints_from_directory - Add comment explaining relationship between hints and extras - Change to load hints hierarchically from working_dir to directory (not just single directory) Per review comments on PR block#5759 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

michaelneale · 2025-11-24T02:24:10Z

crates/goose/src/hints/load_hints.rs

 use crate::hints::import_files::read_referenced_files;

+/// Build a gitignore matcher for the given directory
+pub fn build_gitignore(working_dir: &Path) -> Gitignore {


I don't think we want to take gitignore into account, there was a recent change to address that as it was incorrect

michaelneale · 2025-11-24T02:35:33Z

thanks @johnmatthewtennant a good start. Can you explain how this knows the event when it changes into a dir (ie as part of say some shell tool call?) - as I don't think at the platform level goose necessarily knows exactly when it changes dirs righ? (correct me if wrong).

one simpler way (just a thought - not a deep thought): this could be local to the developer MCP as well, where its tools know what dir they are operating in, and can be aware of hints files and ensure they are loaded/unloaded there (or perhaps signalled back to the load hints platform tool?).

So it is a little subtle, but maybe the solution isn't too hard (ie we could make the tools aware of any adjacent file, and ensure it is loaded always?)

Just a thought - happy to explore more, but this is an important initiative, and I think really really needed for multi module and certainly monorepos, so please - do continue and reach out (discord if you can!)

alexhancock · 2025-11-25T21:05:09Z

crates/goose/src/agents/agent.rs

 pub const MANUAL_COMPACT_TRIGGER: &str = "Please compact this conversation";

+/// Maximum number of turns a directory hint can be idle before pruning
+/// Can be overridden with GOOSE_DYNAMIC_SUBDIRECTORY_HINT_PRUNING_TURNS environment variable


I personally think we wouldn't make it configurable. Let's just decide on good default behavior. Three sounds fine to me.

alexhancock · 2025-11-25T21:09:13Z

crates/goose/src/agents/agent.rs

+                                        .iter()
+                                        .filter_map(|req| {
+                                            if let Ok(call) = &req.tool_call {
+                                                if call.name == "developer__text_editor" {


Part of me wonders if maybe the right way to do this is to expose an interface to let MCP servers conditionally propose additional hint dirs instead of hardcoding to look at tool call names.

Maybe MCP notifications from server to client?

If we had such a thing, developer could do its thing moving around working on files and constantly suggest to the agent to add and remove dirs (or maybe more generalized in suggesting to add content to the system prompt), but other servers could too.

@johnmatthewtennant @michaelneale Thoughts?

I would be curious what claude code, codex do, gemini-cli - latter 2 are open source so if we peek at them, that is probably a reasonable pattern to follow. I think what is important is that it works how people expect, and I can appreciate that you would have hints at different levels (any non trivial app ends up using monorepo patterns) so I think is important we do something here - especially if people expect it to just work (ie we would have things in ui/desktop for goose which are differnet from AGENTS.md in the root? that isn't that surprising).

I think this approach has legs...

@johnmatthewtennant Thoughts on this?

having thought a bit more, I think it does make sense to load the local hints as best it can, it is what I would want to do. Especially when I look at large mono repos - would be the most scalable way to let large teams tweak things. If you end up in their module - their hints should apply for sure. I think same could apply to skills and other things which can be loaded.

michaelneale · 2025-12-16T03:13:11Z

@johnmatthewtennant still interested in this - I think the idea is sound, needs some updating to main though

johnmatthewtennant · 2025-12-17T00:07:22Z

@michaelneale Sorry for the delay! I'm working on updating from main now

Implements dynamic loading of hint files from subdirectories when accessing files in new directories during agent execution. This allows hints to be loaded progressively as the agent navigates through different parts of a monorepo, rather than loading all hints upfront. Key features: - Load hints from directories dynamically when text_editor tool is used - Track loaded directories to avoid redundant loading - Prune idle directory contexts after configurable number of turns - Maintain session state for loaded hints across tool calls

…ctory hint loading Per Michael's comment on load_hints.rs:11, the new dynamic subdirectory hint loading feature should not use gitignore. This was based on the recent change in block#5688 that removed gitignore from the developer MCP. Changes: - Remove build_gitignore() function (NEW code) - Remove gitignore parameter from load_hints_from_directory() (NEW code) - Use empty gitignore in dynamic subdirectory loading to avoid filtering - Keep gitignore in existing load_hint_files() function (EXISTING code) - Keep gitignore in read_referenced_files() function (EXISTING code) - Inline gitignore building in prompt_manager.rs for existing startup hint loading This ensures that only the NEW dynamic subdirectory loading feature ignores gitignore, while the EXISTING startup hint loading continues to respect it. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com>

🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com>

Marks test_hint_loading_security_guards and test_hint_loading_filesystem_updates as serial to prevent SQLite database lock errors when running in parallel in CI. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com>

Apply AGENTS.md code quality rules: - Move 11 unit tests from inline modules to tests/ folder - Improve constant/variable naming for clarity - Downgrade routine info! logs to debug! level - Remove redundant comments that restate code All tests pass. No functional changes. Note: goose-self-test.yaml not updated as the feature already has comprehensive integration tests in dynamic_subdirectory_hints_test.rs 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com>

Restructure the build() method to maintain the original control flow, reducing unnecessary changes. Keeps the same two-step process: build the extras vec first, then sanitize - just extracting .content from the new SystemPromptExtra struct. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com>

Remove comments that restate obvious code per AGENTS.md guidelines: - Remove "This happens on every agent iteration" comment - Remove "Check if directory exists" and "Check if file exists" comments - Remove redundant doc comments on helper methods Revert unnecessary refactoring of existing code: - Restore original loop-based implementation in get_local_directories - Restore original test code ordering These changes reduce the diff by ~24 lines while maintaining the same functionality and focusing the PR on new features rather than refactoring existing working code. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com>

Remove comments that restate function/field names per AGENTS.md: - Remove obvious field and method doc comments - Use unwrap_or_default() instead of unwrap_or_else for idiomaticity Reduces diff by 7 lines while keeping all useful documentation (security notes, return value explanations, behavioral context). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com>

Remove 6 comments that restate what the code obviously does: - "Verify state/hints/loaded" comments before assertions - "Cleanup" comments before delete_session calls Keeps all useful comments that explain test structure, turn context, and expected behaviors per AGENTS.md guidelines. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com>

Remove 4 comments that restate what the code obviously does: - extension_data_test.rs: Remove "Verify context details", "Load directories at different turns", "Access auth at turn 8" - load_hints_test.rs: Remove "Create agents.md with import in subdir" Keeps all useful comments that explain test purpose, expected behaviors, and complex logic per AGENTS.md guidelines. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com>

🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

This function is only used within the hints module, so there's no need to export it publicly. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

This method was only exposed for testing. The test now directly uses LoadedAgentsState.prune_stale() which is the actual logic being tested, reducing the public API surface. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

The test_hint_loading_and_pruning_integration test was missing the #[serial] attribute, causing SQLite database locking errors when running in parallel with other tests. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

johnmatthewtennant · 2025-12-18T00:39:50Z

@michaelneale @alexhancock

I've discovered that this feature is currently incompatible with the code_execution extension (added in #6030, merged Dec 15).

The dynamic hint loading triggers on developer__text_editor tool calls:

const TOOLS_THAT_TRIGGER_HINT_LOADING: &[&str] = &["developer__text_editor"];

However, when code_execution is enabled (now the default), all tool calls are wrapped. Instead of seeing:

developer__text_editor({ path: "subdir/file.py" })

The agent sees:

code_execution__execute_code({ 
  code: "import { text_editor } from 'developer';\ntext_editor({ path: 'subdir/file.py' });" 
})

The file path is buried in JavaScript code, so the trigger never fires.

I need to rethink how we hook into the tool call and capture the path

michaelneale · 2025-12-22T22:53:44Z

gotcha - yeah underneath it is still calling the same dev extension, so there may be a way to trap it deeper. Also, I woudn't necessarily expect that we wont' have other tools as peers to code_execution - ie developer one, so it may still be worthwhile having that in there in the meantime?

michaelneale · 2026-01-21T04:25:22Z

closing as this is a little old - please do re-open when ready

johnmatthewtennant changed the title ~~FIRST DRAFT~~ dynamic subdirectory hint loading Nov 16, 2025

johnmatthewtennant force-pushed the jtennant/dynamic-subdirectory-hint-loading branch from 517a356 to 0e765ba Compare November 16, 2025 18:17

johnmatthewtennant mentioned this pull request Nov 21, 2025

Dynamically load hints in subdirectories #5840

Open

1 task

michaelneale reviewed Nov 24, 2025

View reviewed changes

michaelneale self-assigned this Nov 24, 2025

michaelneale added p2 Priority 2 - Medium large Weight label for Hacktoberfest 2025 issues labels Nov 24, 2025

michaelneale assigned alexhancock Nov 24, 2025

alexhancock reviewed Nov 25, 2025

View reviewed changes

michaelneale added enhancement New feature or request core Pertains to core goose functionality labels Nov 25, 2025

johnmatthewtennant force-pushed the jtennant/dynamic-subdirectory-hint-loading branch from 1778d89 to 4a06421 Compare December 16, 2025 22:28

johnmatthewtennant force-pushed the jtennant/dynamic-subdirectory-hint-loading branch 2 times, most recently from 0a811b5 to 273a82c Compare December 17, 2025 18:00

johnmatthewtennant and others added 5 commits December 17, 2025 13:19

Fix integration test after gitignore removal

30c3e9e

🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com>

johnmatthewtennant force-pushed the jtennant/dynamic-subdirectory-hint-loading branch from 273a82c to 1e7ad87 Compare December 17, 2025 18:19

johnmatthewtennant and others added 5 commits December 17, 2025 13:31

johnmatthewtennant and others added 5 commits December 17, 2025 14:21

refactor: revert accidental test line reordering

819e5fc

🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

refactor: keep find_git_root private

38f9ba0

This function is only used within the hints module, so there's no need to export it publicly. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

re-add deleted whitespace

3b269fb

johnmatthewtennant force-pushed the jtennant/dynamic-subdirectory-hint-loading branch from ae867b4 to 3b269fb Compare December 17, 2025 20:09

johnmatthewtennant added 3 commits December 17, 2025 16:18

refactor: track dynamic hints in-memory (no session persistence)

ca66271

refactor: drop persisted dynamic-hint state from ExtensionData

86aaf3b

test: dedupe dynamic hint loader tests

237ab14

michaelneale closed this Jan 21, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

dynamic subdirectory hint loading#5759

dynamic subdirectory hint loading#5759
johnmatthewtennant wants to merge 18 commits intoblock:mainfrom
johnmatthewtennant:jtennant/dynamic-subdirectory-hint-loading

johnmatthewtennant commented Nov 16, 2025 •

edited

Loading

Uh oh!

michaelneale Nov 24, 2025

Uh oh!

michaelneale commented Nov 24, 2025

Uh oh!

alexhancock Nov 25, 2025

Uh oh!

alexhancock Nov 25, 2025

Uh oh!

michaelneale Nov 25, 2025

Uh oh!

alexhancock Dec 12, 2025

Uh oh!

michaelneale Dec 13, 2025

Uh oh!

michaelneale commented Dec 16, 2025

Uh oh!

johnmatthewtennant commented Dec 17, 2025

Uh oh!

johnmatthewtennant commented Dec 18, 2025 •

edited

Loading

Uh oh!

michaelneale commented Dec 22, 2025

Uh oh!

michaelneale commented Jan 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

johnmatthewtennant commented Nov 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Type of Change

AI Assistance

Testing

Related Issues

Uh oh!

michaelneale Nov 24, 2025

Choose a reason for hiding this comment

Uh oh!

michaelneale commented Nov 24, 2025

Uh oh!

alexhancock Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

alexhancock Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

michaelneale Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

alexhancock Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

michaelneale Dec 13, 2025

Choose a reason for hiding this comment

Uh oh!

michaelneale commented Dec 16, 2025

Uh oh!

johnmatthewtennant commented Dec 17, 2025

Uh oh!

johnmatthewtennant commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

michaelneale commented Dec 22, 2025

Uh oh!

michaelneale commented Jan 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

johnmatthewtennant commented Nov 16, 2025 •

edited

Loading

johnmatthewtennant commented Dec 18, 2025 •

edited

Loading