feat: Structured output for recipes by jsibbison-square · Pull Request #3188 · block/goose

jsibbison-square · 2025-07-01T05:14:07Z

Main Changes

Adds a new response.json_schema key in recipe yaml.
When a json schema is described here then the agent will call a tool with its final output that matches that schema (otherwise a validation error will prompt the agent to correct its tool use).
The final message to the user will be a single line json of this final output, making it easier to extract it (its just the last line)
This is only applicable to goose run with recipes currently.

Other Details

The response.json_schema is validated using jsonschema crate
If the agent tries to finish and it has not called the final tool for its output we send the agent a new message directing it to use the final_tool to output the expected message

This PR does not add support to use structured output when running a recipe that defines it via the Desktop app. Future PR.

Example
Recipe

version: 1.0.0
title: "structured output"
description: "Create a structured output"
prompt: "Begin. We have the items: Fish, Spoon and Dog."
response:
  json_schema:
    type: object
    properties:
      total:
        type: number
        description: "The total number of items"
      num_characters:
        type: number
        description: "The number of characters in the items"
      first_item:
        type: string
        description: "The first item"
      last_item:
        type: string
        description: "The last item"
      items:
        type: array
        items:
          type: string
          description: "Each of the items"
    required:
      - total
      - num_characters
      - first_item
      - last_item
      - items

Usage

Loading recipe: structured output
Description: Create a structured output

starting session | provider: databricks model: databricks-claude-3-7-sonnet
    logging to /.../.local/share/goose/sessions/20250701_092723.jsonl
    working directory: /.../Development/goose/structured-output
I'll help you process these items and provide the structured output according to the required schema.

Let me analyze your items:
- Fish
- Spoon
- Dog

Now I'll prepare the JSON output with the required fields:
1. `total`: The total number of items (3)
2. `num_characters`: The total number of characters in all items (12 - "Fish" has 4, "Spoon" has 5, and "Dog" has 3)
3. `first_item`: The first item in the list ("Fish")
4. `last_item`: The last item in the list ("Dog")
5. `items`: An array containing all items (["Fish", "Spoon", "Dog"])

Let me submit this as the final output:
─── final_output |  ──────────────────────────
final_output: ...


◇  Goose would like to call the above tool, do you allow?
│  Allow
│


{"first_item":"Fish","items":["Fish","Spoon","Dog"],"last_item":"Dog","num_characters":12,"total":3}

…ructured-output * origin/main: (22 commits) feat(desktop): Prioritize suffix when truncating path in header (#3110) chore(release): release version 1.0.31 (#3185) feat: additional sub recipes via command line (#3163) Add Internal Recipes To Recipes Cookbook (#3179) pipe the argument to storage (#3184) docs: removing comment (#3183) docs: add generator option to create recipe (#3182) update the path for temporal (#3131) docs: add link to Square MCP (#3181) attempt to fix build #3 (#3180) attempt fix folde permissions for windows build (#3178) attempt to fix windows cli permission issue (#3177) allow to use dev/null for no-session mode (#3176) feat: change naming of recipe creation in more menu links (#3175) Docs: Add Recipe video to landing page (#3173) Docs: Create new directory when starting new session (#3174) fixes cron parsing issues (#3172) fix: handle Windows package subdirectory in CLI installation script (#3171) fixed the npx/uvx content (#3170) Mark helper scripts as executable (#3169) ...

DOsinga

can you also update the recipe documentation?

DOsinga · 2025-07-01T16:46:47Z

crates/goose/src/agents/agent.rs

            return (request_id, Ok(ToolCallResult::from(result)));
        }

+        if tool_call.name == FINAL_OUTPUT_TOOL_NAME {


oof this function is a bit of a mess! your addition is of course in line with what the rest looks like, but we should fix this at some point so if you feel the itch ...

DOsinga · 2025-07-01T16:55:44Z

crates/goose/src/recipe/mod.rs

+pub struct Response {
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub json_schema: Option<serde_json::Value>,
+}


do you have a strong preference about making this nested? in the ticket we talk about just having a output_json_schema field

yeah I think we should leave ourselves open to alternative response formats and extra controls (I expect someone will ask for a 'max_attempts' option to prevent unlimited attempts at some stage... I'm building a MAX_TURNS option into goose atm which I think is a better version).

DOsinga · 2025-07-01T16:58:41Z

crates/goose/src/agents/final_output_tool.rs

+                    }
+                },
+                "required": ["final_output"]
+            }),


couldn't we just insert the json schema at this point? asking the llm to do it in this format probably works, but since the tool takes a json schema, why not use that?

🔥 Very nice, that cleans up the code a bit too.

DOsinga · 2025-07-01T20:55:37Z

crates/goose/src/agents/final_output_tool.rs

+                    "Invalid JSON format: {}\n\nExpected format:\n{}\n\nPlease provide valid JSON that matches the expected schema.",
+                    e,
+                    serde_json::to_string_pretty(self.response.json_schema.as_ref().unwrap()).unwrap_or_else(|_| "Invalid schema".to_string())
+                ));


does it recover from this?

yes. I managed to fudge the agent to do it wrong the first time to test with goose. I've implemented this pattern a few times and it has worked pretty well.

DOsinga · 2025-07-01T20:59:41Z

crates/goose/src/agents/final_output_tool.rs

+    }
+}
+
+#[cfg(test)]


there are a lot of tests here. I would suggest to remove the ones that will never find a bug. the tests in the recipe mod feel more useful

Old TDD habits 🙃 . Removing some.

github-actions · 2025-07-01T23:26:39Z

PR Preview Action v1.6.0
Preview removed because the pull request was closed.
2025-07-02 02:18 UTC

…pping in a param

* main: feat: Structured output for recipes (#3188) Fix cost tracking accuracy and OpenRouter model pricing (#3189) docs: update cli install instructions for windows (#3205) Docs: Cost tracking on the desktop app (#3204) feat: Adding streamable-http transport support for backend, desktop and cli (#2942) fix: use the correct `contains` syntax on create-recipe-pr.yml (#3193)

* main: fix: Pass Google AI API key in HTTP header, not query param (block#3192) docs: add linter to CONTRIBUTING.md (block#3168) feat: Structured output for recipes (block#3188) Fix cost tracking accuracy and OpenRouter model pricing (block#3189) docs: update cli install instructions for windows (block#3205) Docs: Cost tracking on the desktop app (block#3204) feat: Adding streamable-http transport support for backend, desktop and cli (block#2942) fix: use the correct `contains` syntax on create-recipe-pr.yml (block#3193)

* main: (37 commits) fix: fix desktop recipe url generation (block#3209) feat: improve UX for saving recipes (block#3214) fix: Pass Google AI API key in HTTP header, not query param (block#3192) docs: add linter to CONTRIBUTING.md (block#3168) feat: Structured output for recipes (block#3188) Fix cost tracking accuracy and OpenRouter model pricing (block#3189) docs: update cli install instructions for windows (block#3205) Docs: Cost tracking on the desktop app (block#3204) feat: Adding streamable-http transport support for backend, desktop and cli (block#2942) fix: use the correct `contains` syntax on create-recipe-pr.yml (block#3193) Temporarily Remove GH Copilot Provider (block#3199) docs: fix tab navigation (block#3201) feat: use tiktoken-rs instead of tokenizers, single global tokenizer (block#3115) add playwright-mcp server to extensions list (block#3010) Add `/extension` path for extension installation (block#3011) feat(desktop): Prioritize suffix when truncating path in header (block#3110) chore(release): release version 1.0.31 (block#3185) feat: additional sub recipes via command line (block#3163) Add Internal Recipes To Recipes Cookbook (block#3179) pipe the argument to storage (block#3184) ...

* main: (150 commits) Defend against invalid sessions (block#3229) Clean up session file optionality for --no-session (block#3230) Feat: Support Recipe Parameters in Goose desktop app (block#3155) docs: update recipe example (block#3222) Add native OAuth 2.0 authentication support to MCP client (block#3213) build: Check in Cargo.lock changes (block#3220) fix: fix desktop recipe url generation (block#3209) feat: improve UX for saving recipes (block#3214) fix: Pass Google AI API key in HTTP header, not query param (block#3192) docs: add linter to CONTRIBUTING.md (block#3168) feat: Structured output for recipes (block#3188) Fix cost tracking accuracy and OpenRouter model pricing (block#3189) docs: update cli install instructions for windows (block#3205) Docs: Cost tracking on the desktop app (block#3204) feat: Adding streamable-http transport support for backend, desktop and cli (block#2942) fix: use the correct `contains` syntax on create-recipe-pr.yml (block#3193) Temporarily Remove GH Copilot Provider (block#3199) docs: fix tab navigation (block#3201) feat: use tiktoken-rs instead of tokenizers, single global tokenizer (block#3115) add playwright-mcp server to extensions list (block#3010) ...

Signed-off-by: Adam Tarantino <tarantino.adam@gmail.com>

Signed-off-by: Soroosh <soroosh.sarabadani@gmail.com>

Signed-off-by: Kyle Santiago <kyle@privkey.io>

jsibbison-square added 10 commits June 30, 2025 13:10

Add response.json_schema to recipe struct

6b6d85c

Structured output for cli

daa0950

final_output_tool new tests

ca9c008

final_output_tool tool call tests

0ad442a

builder change test

5d87155

Adds final output tool add test

a759d8f

Add test for finding the named tool

2073092

Agent final tool call last output message

a84fef2

cargo fmt

7dd3e16

jsibbison-square requested a review from DOsinga July 1, 2025 05:14

clippy fixes

6d850d7

jsibbison-square changed the title ~~WIP: structured output for recipes~~ feat: Structured output for recipes Jul 1, 2025

jsibbison-square marked this pull request as ready for review July 1, 2025 09:30

jsibbison-square mentioned this pull request Jul 1, 2025

feat: Adds run --porcelain cli option to output only the final text message to stdout #3106

Closed

DOsinga approved these changes Jul 1, 2025

View reviewed changes

Updates recipe doco with response.json_schema

12c5c36

jsibbison-square added 2 commits July 2, 2025 11:28

Reduce some less valuable tests

fb77452

Use json schema as the final_output_tool input_schema rather than wra…

237876b

…pping in a param

jsibbison-square merged commit 0a00b0f into main Jul 2, 2025
9 checks passed

jsibbison-square deleted the jsibbison-20250630-structured-output branch July 2, 2025 02:16

jsibbison-square mentioned this pull request Jul 2, 2025

feat: GUI can use structured output present in recipes #3218

Merged

jsibbison-square mentioned this pull request Jul 3, 2025

Adds json schema validation to goose recipe validate cli #3234

Merged

chaitanyarahalkar mentioned this pull request Jul 5, 2025

Macos-only Sandboxing chaitanyarahalkar/goose#9

Open

atarantino pushed a commit to atarantino/goose that referenced this pull request Jul 14, 2025

feat: Structured output for recipes (block#3188)

e02e892

Signed-off-by: Adam Tarantino <tarantino.adam@gmail.com>

s-soroosh pushed a commit to s-soroosh/goose that referenced this pull request Jul 18, 2025

feat: Structured output for recipes (block#3188)

f9bc421

Signed-off-by: Soroosh <soroosh.sarabadani@gmail.com>

kwsantiago pushed a commit to kwsantiago/goose that referenced this pull request Jul 19, 2025

feat: Structured output for recipes (block#3188)

56e6598

Signed-off-by: Kyle Santiago <kyle@privkey.io>

cbruyndoncx pushed a commit to cbruyndoncx/goose that referenced this pull request Jul 20, 2025

feat: Structured output for recipes (block#3188)

e1debd7

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Structured output for recipes#3188

feat: Structured output for recipes#3188
jsibbison-square merged 14 commits intomainfrom
jsibbison-20250630-structured-output

jsibbison-square commented Jul 1, 2025 •

edited

Loading

Uh oh!

DOsinga left a comment

Uh oh!

DOsinga Jul 1, 2025

Uh oh!

DOsinga Jul 1, 2025

Uh oh!

jsibbison-square Jul 1, 2025

Uh oh!

DOsinga Jul 1, 2025

Uh oh!

jsibbison-square Jul 2, 2025

Uh oh!

DOsinga Jul 1, 2025

Uh oh!

jsibbison-square Jul 1, 2025 •

edited

Loading

Uh oh!

DOsinga Jul 1, 2025

Uh oh!

jsibbison-square Jul 2, 2025

Uh oh!

github-actions bot commented Jul 1, 2025 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

jsibbison-square commented Jul 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DOsinga left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jsibbison-square Jul 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jul 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jsibbison-square commented Jul 1, 2025 •

edited

Loading

jsibbison-square Jul 1, 2025 •

edited

Loading

github-actions bot commented Jul 1, 2025 •

edited

Loading