test: fix recipe and audio tests to avoid side effects by codefromthecrypt · Pull Request #6231 · block/goose

codefromthecrypt · 2025-12-22T11:25:08Z

Summary

Fix tests that cause side effects during cargo test:

recipe.rs: Tests were launching Goose Desktop via open::that(). Added injectable opener pattern with handle_open_with() so tests can mock the opener and verify URLs without launching external apps. Added failure case tests.
audio.rs: Test was triggering keychain access. Added EnvVarGuard for tokio-safe env var manipulation to set OPENAI_API_KEY before AppState::new().

Type of Change

Tests

AI Assistance

This PR was created or reviewed with AI assistance

Testing

cargo test passes with no keychain popups or Goose Desktop launches
Added test_handle_open_opener_fails and test_handle_open_invalid_recipe to verify error handling paths

Related Issues

Completes #6219

- recipe.rs: Use injectable opener pattern to prevent tests from launching Goose Desktop. Added handle_open_with() with injectable opener and output writer. Created run_handle_open() test helper. Added failure case tests for opener errors and invalid recipes. - audio.rs: Use EnvVarGuard for tokio-safe env var manipulation in test_transcribe_endpoint_requires_auth to avoid keychain access. Signed-off-by: Adrian Cole <adrian@tetrate.io>

codefromthecrypt · 2025-12-22T11:47:13Z

ran cargo test manually 3 times in a row and no keychain requests

DOsinga · 2025-12-22T14:38:09Z

crates/goose-cli/src/commands/recipe.rs

        let params = vec!["name=Alice".to_string(), "role=developer".to_string()];
-        let result = handle_open(&recipe_path, &params);
-        // The result may be Ok or Err depending on whether the system can open the URL
-        // In a test environment, it will likely fail to open, but that's fine


this sounds like an LLM wanting to write a test but not knowing how to

crates/goose-server/src/routes/audio.rs

Copilot

Pull request overview

This PR fixes test side effects by preventing external system interactions during test execution. The main changes involve migrating from a custom EnvVarGuard to the env-lock crate for thread-safe environment variable management, and introducing dependency injection for the recipe opener to avoid launching Goose Desktop during tests.

Replaced custom EnvVarGuard with env-lock crate for tokio-safe environment variable management
Added injectable opener pattern to recipe.rs to mock external app launches in tests
Set OPENAI_API_KEY before AppState::new() in audio tests to prevent keychain prompts

Reviewed changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
crates/goose/Cargo.toml	Added `env-lock` as dev dependency for test environment management
crates/goose-server/Cargo.toml	Added `env-lock` as dev dependency for test environment management
crates/goose/src/providers/factory.rs	Migrated from custom `EnvVarGuard` to `env-lock::lock_env` for thread-safe env var handling in tests
crates/goose-server/src/routes/audio.rs	Added env var setup before `AppState::new()` to prevent keychain access during tests
crates/goose-cli/src/commands/recipe.rs	Extracted `handle_open_with` to enable dependency injection of opener function, added tests for error paths
Cargo.lock	Updated lock file with `env-lock` v1.0.1 dependency

crates/goose/src/providers/factory.rs

codefromthecrypt · 2025-12-23T01:26:04Z

crates/goose/src/providers/factory.rs

-        if let Ok(limit) = limit_str.parse::<usize>() {
-            worker_config = worker_config.with_context_limit(Some(limit));
-        }
+    if let Ok(limit) = global_config.get_param::<usize>("GOOSE_WORKER_CONTEXT_LIMIT") {


unrelated bug but fixed while here. We didn't actually read anything in our previous tests so they were not really helpful before

Signed-off-by: Adrian Cole <adrian@tetrate.io>

Copilot

Pull request overview

Copilot reviewed 7 out of 8 changed files in this pull request and generated no new comments.

codefromthecrypt · 2025-12-23T02:14:37Z

crates/goose/src/providers/factory.rs

-        let result = create("openai", ModelConfig::new_or_fail("gpt-4o-mini")).await;
-
-        match result {
-            Ok(_) => {}


rewrote all the assertions here as they did nothing

* main: (155 commits) remove Tool Selection Strategy preview (#6250) fix(cli): correct bash syntax in terminal integration functions (#6181) fix : opening a session to view it modifies session history order in desktop (#6156) test: fix recipe and audio tests to avoid side effects (#6231) chore: Update gemini versions in test_providers.sh (#6246) feat: option to stream json - jsonl really (#6228) feat: add mcp app renderer (#6095) docs: update skills extension to support .agents/skills directories (#6199) Add YouTube short to Chrome DevTools MCP tutorial (#6244) docs: Caveats for privacy information in logs documentation (#6218) move goose issue solver to opus (#6233) feat: improved UX for tool calls via execute_code (#6205) Blog: Code Mode Doesn't Replace MCP (#6227) fix: prevent keychain requests during cargo test (#6219) test: fix test_max_turns_limit slow execution and wrong message type (#6221) Skills vs MCP blog (#6220) Add blog post: Does Your AI Agent Need a Plan? (#6209) fix(ui): enable MCP UI to send a prompt message when an element is clicked (#6207) docs: param option for recipe deeplink/open (#6206) docs: edit in place or fork session (#6203) ...

codefromthecrypt mentioned this pull request Dec 22, 2025

fix: prevent keychain requests during cargo test #6219

Merged

2 tasks

codefromthecrypt force-pushed the fix-recipe-audio-tests branch from 3d82f17 to c006038 Compare December 22, 2025 11:35

codefromthecrypt marked this pull request as ready for review December 22, 2025 11:46

DOsinga approved these changes Dec 22, 2025

View reviewed changes

Copilot AI review requested due to automatic review settings December 23, 2025 00:42

Copilot started reviewing on behalf of codefromthecrypt December 23, 2025 00:42 View session

Copilot AI reviewed Dec 23, 2025

View reviewed changes

crates/goose/src/providers/factory.rs Outdated Show resolved Hide resolved

codefromthecrypt force-pushed the fix-recipe-audio-tests branch from 1550098 to da35fdc Compare December 23, 2025 01:24

codefromthecrypt commented Dec 23, 2025

View reviewed changes

env-lock

db34962

Signed-off-by: Adrian Cole <adrian@tetrate.io>

Copilot AI review requested due to automatic review settings December 23, 2025 02:09

codefromthecrypt force-pushed the fix-recipe-audio-tests branch from da35fdc to db34962 Compare December 23, 2025 02:09

Copilot started reviewing on behalf of codefromthecrypt December 23, 2025 02:10 View session

Copilot AI reviewed Dec 23, 2025

View reviewed changes

codefromthecrypt commented Dec 23, 2025

View reviewed changes

codefromthecrypt merged commit 2f87672 into block:main Dec 23, 2025
24 checks passed

codefromthecrypt deleted the fix-recipe-audio-tests branch December 23, 2025 02:26

github-actions bot mentioned this pull request Jan 6, 2026

chore(release): release version 1.19.0 (minor) #6344

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test: fix recipe and audio tests to avoid side effects#6231

test: fix recipe and audio tests to avoid side effects#6231
codefromthecrypt merged 2 commits intoblock:mainfrom
codefromthecrypt:fix-recipe-audio-tests

codefromthecrypt commented Dec 22, 2025

Uh oh!

codefromthecrypt commented Dec 22, 2025

Uh oh!

DOsinga Dec 22, 2025

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

codefromthecrypt Dec 23, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

codefromthecrypt Dec 23, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

codefromthecrypt commented Dec 22, 2025

Summary

Type of Change

AI Assistance

Testing

Related Issues

Uh oh!

codefromthecrypt commented Dec 22, 2025

Uh oh!

DOsinga Dec 22, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

codefromthecrypt Dec 23, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

codefromthecrypt Dec 23, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants