Regenerate canonical models when release branch is created. by katzdave · Pull Request #6127 · block/goose

katzdave · 2025-12-15T22:00:40Z

How this will work:

Runs build_canonical_models -> creates canonical_models.json
Runs canonical_model_checker -> creates canonical_mapping_report.json

canonical_model_checker needs secrets to access our common providers.

canonical_mapping_report.json isn't that useful, but what canonical_model_checker prints to stdout about which model mappings were changed/added is, so we might want to expose that a little better.

Can discuss more, but I think on release is a reasonable cadence to update the canonical model mapping + gives us a chance to review if anything catastrophic happened.

Copilot

Pull request overview

This PR automates the regeneration of canonical model mappings during the release process. The changes ensure that whenever a release branch is created (minor or patch), the canonical models and their mappings are automatically updated and committed.

Key Changes:

Canonical model generation is now integrated into the release preparation workflow
API keys for major providers are passed through GitHub Actions workflows to enable model checking

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
Justfile	Adds canonical model generation and checking steps to the prepare-release target, staging the generated JSON files for commit
.github/workflows/patch-release.yaml	Passes provider API keys as secrets to the reusable release workflow
.github/workflows/minor-release.yaml	Passes provider API keys as secrets to the reusable release workflow
.github/workflows/create-release-pr.yaml	Declares required secrets and exposes them as environment variables during release branch creation

Copilot · 2025-12-15T22:01:03Z

.github/workflows/create-release-pr.yaml

+        required: false
+      OPENAI_API_KEY:
+        required: false
+      GOOGLE_API_KEY:
+        required: false
+      OPENROUTER_API_KEY:
+        required: false
+      XAI_API_KEY:
+        required: false
+      TETRATE_API_KEY:
+        required: false


API keys should be marked as required: true since canonical_model_checker depends on them to function correctly. Without these secrets, the release process will silently fail to generate accurate canonical mappings.

Suggested change

required: false

OPENAI_API_KEY:

required: false

GOOGLE_API_KEY:

required: false

OPENROUTER_API_KEY:

required: false

XAI_API_KEY:

required: false

TETRATE_API_KEY:

required: false

required: true

OPENAI_API_KEY:

required: true

GOOGLE_API_KEY:

required: true

OPENROUTER_API_KEY:

required: true

XAI_API_KEY:

required: true

TETRATE_API_KEY:

required: true

Copilot · 2025-12-15T22:01:03Z

Justfile

+    @cargo run --example build_canonical_models
+    @cargo run --example canonical_model_checker


These commands will fail in CI if the required API keys are not available. Consider adding error handling or checks to ensure the commands succeed before staging their output files.

jamadeo

Makes sense to me to put this here. How likely do you think it is to fail? If it fails, the release PR never gets created, which is not great but if it's pretty reliable that's good enough.

I don't think these should be "examples" though. You can create regular binary targets with like this:

goose/crates/goose-server/Cargo.toml

Line 57 in ae8ecbe

[[bin]]

and then run with cargo run --bin build_canonical_models

jamadeo · 2025-12-16T20:36:32Z

Justfile

-    @git add Cargo.toml Cargo.lock ui/desktop/package.json ui/desktop/package-lock.json ui/desktop/openapi.json
+    @cargo run --example build_canonical_models
+    @cargo run --example canonical_model_checker
+    @git add Cargo.toml Cargo.lock ui/desktop/package.json ui/desktop/package-lock.json ui/desktop/openapi.json crates/goose/src/providers/canonical/data/canonical_models.json crates/goose/src/providers/canonical/data/canonical_mapping_report.json


probably time to break this up like

@git add \ Cargo.toml \ Cargo.lock \ ... etc

katzdave · 2025-12-17T19:49:05Z

Makes sense to me to put this here. How likely do you think it is to fail? If it fails, the release PR never gets created, which is not great but if it's pretty reliable that's good enough.

I don't think these should be "examples" though. You can create regular binary targets with like this:

goose/crates/goose-server/Cargo.toml

Line 57 in ae8ecbe

[[bin]]

and then run with cargo run --bin build_canonical_models

Done. Made a new binary and combined these.

Only reasons it would fail are:

Network issues openrouter/breaking schema change
Missing secret for the checker

Should both generally be fine and I'd want to know if it breaks.

Copilot

Pull request overview

Copilot reviewed 9 out of 11 changed files in this pull request and generated 1 comment.

Copilot · 2025-12-18T16:07:13Z

crates/goose/Cargo.toml

 include_dir = "0.7.4"
 tiktoken-rs = "0.6.0"
 chrono = { version = "0.4.38", features = ["serde"] }
+clap = { version = "4.4", features = ["derive"] }


The clap dependency is added but it's duplicated - clap is already in the workspace dependencies. Consider using workspace = true instead of specifying the version directly.

Suggested change

clap = { version = "4.4", features = ["derive"] }

clap = { workspace = true, features = ["derive"] }

* 'main' of github.com:block/goose: (28 commits) Clean PR preview sites from gh-pages branch history (#6161) fix: make goose reviewer less sycophantic (#6171) revert /reply to previous behavior (replacing session history) when full conversation provided (#6058) chore: manually update version (#6166) Integrate pricing with canonical model (#6130) Regenerate canonical models when release branch is created. (#6127) fix: use correct parameter name in read_module handler (#6148) docs: blog for code mode MCP (#6126) test: add ACP integration test (#6150) docs: auto download updates (#6163) fix: respect default_enabled value of platform extensions (#6159) docs: skills (#6062) fix: add conditional configuration for GOOSE_BIN_DIR in PATH (#5940) Update dependencies to help in Fedora packaging (#5835) fix: make goose reviewer less bad (#6154) docs: create/edit recipe button (#6145) fix(google): Fix 400 Bad Request error with Gemini 3 thought signatures (#6035) fix: we don't need to warn about tool count when in code mode (#6149) deps: upgrade agent-client-protocol to 0.9.0 (#6109) fix(providers): fix for gemini-cli on windows to work around cmd's multiline prompt limitations #5911 (#5966) ... # Conflicts: # ui/desktop/src/api/sdk.gen.ts # ui/desktop/src/hooks/useAgent.ts

Signed-off-by: Brandon Kvarda <brandon.kvarda@databricks.com>

katzdave added 2 commits December 15, 2025 16:47

run canonical model refresh on prepare-release

efa1098

pass thru secrets

04fcfaa

katzdave requested review from Copilot and jamadeo December 15, 2025 22:00

Copilot AI reviewed Dec 15, 2025

View reviewed changes

jamadeo approved these changes Dec 16, 2025

View reviewed changes

katzdave added 4 commits December 17, 2025 14:37

combine canonical model scripts + separate build path

22914be

update bin

8124094

rerun the script

84b940f

clean nl

9797475

no-check flag

28711f2

Copilot AI review requested due to automatic review settings December 18, 2025 16:04

Copilot started reviewing on behalf of katzdave December 18, 2025 16:05 View session

Copilot AI reviewed Dec 18, 2025

View reviewed changes

rerun script again

2117494

katzdave merged commit 473f269 into main Dec 18, 2025
18 checks passed

katzdave deleted the dkatz/run-gen-canonical branch December 18, 2025 16:44

dianed-square pushed a commit that referenced this pull request Dec 19, 2025

Regenerate canonical models when release branch is created. (#6127)

346bfdf

bkvarda pushed a commit to bkvarda/goose that referenced this pull request Dec 19, 2025

Regenerate canonical models when release branch is created. (block#6127)

468f811

Signed-off-by: Brandon Kvarda <brandon.kvarda@databricks.com>

github-actions bot mentioned this pull request Dec 19, 2025

chore(release): release version 1.18.0 (minor) #6194

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Regenerate canonical models when release branch is created.#6127

Regenerate canonical models when release branch is created.#6127
katzdave merged 8 commits intomainfrom
dkatz/run-gen-canonical

katzdave commented Dec 15, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Dec 15, 2025

Uh oh!

Copilot AI Dec 15, 2025

Uh oh!

jamadeo left a comment

Uh oh!

jamadeo Dec 16, 2025

Uh oh!

katzdave Dec 17, 2025

Uh oh!

katzdave commented Dec 17, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Dec 18, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		@cargo run --example build_canonical_models
		@cargo run --example canonical_model_checker

	clap = { version = "4.4", features = ["derive"] }
	clap = { workspace = true, features = ["derive"] }

Conversation

katzdave commented Dec 15, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

jamadeo left a comment

Choose a reason for hiding this comment

Uh oh!

jamadeo Dec 16, 2025

Choose a reason for hiding this comment

Uh oh!

katzdave Dec 17, 2025

Choose a reason for hiding this comment

Uh oh!

katzdave commented Dec 17, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Dec 18, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants