test: add model-specific snapshots for coreTools by aishaneeshah · Pull Request #18707 · google-gemini/gemini-cli

aishaneeshah · 2026-02-10T03:55:16Z

Summary

Adds model-specific snapshots for core tools to enable manual inspection of tool descriptions and schemas for different model families (specifically gemini-2.5-pro and gemini-3-pro-preview).

Details

Created a new test file packages/core/src/tools/definitions/coreToolsModelSnapshots.test.ts.
Uses standard Vitest toMatchSnapshot() to generate a single .snap file, ensuring consistency with other tests in the repository (e.g., prompts.test.ts).
Mocks node:os and process.platform to 'linux' in the test to ensure deterministic, cross-platform snapshots.
Snapshots include read_file, write_file, grep_search, glob, list_directory, and run_shell_command.
Formatting is aligned with standard Vitest output, which includes multi-line arrays for required parameters for better readability.

Related Issues

Related to #17958

How to Validate

Run the workspace-specific test:

npm test -w @google/gemini-cli-core -- src/tools/definitions/coreToolsModelSnapshots.test.ts

Expected result: 12 tests pass, and snapshots are verified in packages/core/src/tools/definitions/__snapshots__/coreToolsModelSnapshots.test.ts.snap.

Pre-Merge Checklist

Updated relevant documentation and README (if needed)
Added/updated tests (if needed)
Noted breaking changes (if any)
Validated on required platforms/methods:
- MacOS (via CI)
- Windows (via CI)
- Linux (Local + CI)

gemini-cli · 2026-02-10T03:55:25Z

Hi there! Thank you for your contribution to Gemini CLI.

To improve our contribution process and better track changes, we now require all pull requests to be associated with an existing issue, as announced in our recent discussion and as detailed in our CONTRIBUTING.md.

This pull request is being closed because it is not currently linked to an issue. Once you have updated the description of this PR to link an issue (e.g., by adding Fixes #123 or Related to #123), it will be automatically reopened.

How to link an issue:
Add a keyword followed by the issue number (e.g., Fixes #123) in the description of your pull request. For more details on supported keywords and how linking works, please refer to the GitHub Documentation on linking pull requests to issues.

Thank you for your understanding and for being a part of our community!

gemini-cli · 2026-02-10T03:55:27Z

Hi @aishaneeshah, thank you so much for your contribution to Gemini CLI! We really appreciate the time and effort you've put into this.

We're making some updates to our contribution process to improve how we track and review changes. Please take a moment to review our recent discussion post: Improving Our Contribution Process & Introducing New Guidelines.

Key Update: Starting January 26, 2026, the Gemini CLI project will require all pull requests to be associated with an existing issue. Any pull requests not linked to an issue by that date will be automatically closed.

Thank you for your understanding and for being a part of our community!

gemini-code-assist · 2026-02-10T03:55:36Z

Summary of Changes

Hello @aishaneeshah, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the testing infrastructure by introducing model-specific snapshots for core tools. This change provides a robust mechanism to ensure the consistency and correctness of tool definitions and their schemas across different Gemini model families, facilitating easier verification and maintenance of tool capabilities.

Highlights

Model-Specific Snapshots: Introduced model-specific snapshots for core tools, enabling manual inspection of tool descriptions and schemas for gemini-2.5-pro and gemini-3-pro-preview.
New Test File: Added a new test file, packages/core/src/tools/definitions/coreToolsModelSnapshots.test.ts, to manage and verify these snapshots.
Snapshot Organization: Snapshots are now organized into a directory structure: __snapshots__/<modelId>/<toolName>.json.
Deterministic Snapshots: Mocked node:os in the test to ensure deterministic (Linux-based) snapshots across different operating systems.
Covered Tools: Snapshots include read_file, write_file, grep_search, glob, list_directory, and run_shell_command.

Changelog

packages/core/src/tools/definitions/snapshots/gemini-2.5-pro/glob.json
- Added a JSON snapshot for the glob tool for the gemini-2.5-pro model.
packages/core/src/tools/definitions/snapshots/gemini-2.5-pro/grep_search.json
- Added a JSON snapshot for the grep_search tool for the gemini-2.5-pro model.
packages/core/src/tools/definitions/snapshots/gemini-2.5-pro/list_directory.json
- Added a JSON snapshot for the list_directory tool for the gemini-2.5-pro model.
packages/core/src/tools/definitions/snapshots/gemini-2.5-pro/read_file.json
- Added a JSON snapshot for the read_file tool for the gemini-2.5-pro model.
packages/core/src/tools/definitions/snapshots/gemini-2.5-pro/run_shell_command.json
- Added a JSON snapshot for the run_shell_command tool for the gemini-2.5-pro model.
packages/core/src/tools/definitions/snapshots/gemini-2.5-pro/write_file.json
- Added a JSON snapshot for the write_file tool for the gemini-2.5-pro model.
packages/core/src/tools/definitions/snapshots/gemini-3-pro-preview/glob.json
- Added a JSON snapshot for the glob tool for the gemini-3-pro-preview model.
packages/core/src/tools/definitions/snapshots/gemini-3-pro-preview/grep_search.json
- Added a JSON snapshot for the grep_search tool for the gemini-3-pro-preview model.
packages/core/src/tools/definitions/snapshots/gemini-3-pro-preview/list_directory.json
- Added a JSON snapshot for the list_directory tool for the gemini-3-pro-preview model.
packages/core/src/tools/definitions/snapshots/gemini-3-pro-preview/read_file.json
- Added a JSON snapshot for the read_file tool for the gemini-3-pro-preview model.
packages/core/src/tools/definitions/snapshots/gemini-3-pro-preview/run_shell_command.json
- Added a JSON snapshot for the run_shell_command tool for the gemini-3-pro-preview model.
packages/core/src/tools/definitions/snapshots/gemini-3-pro-preview/write_file.json
- Added a JSON snapshot for the write_file tool for the gemini-3-pro-preview model.
packages/core/src/tools/definitions/coreToolsModelSnapshots.test.ts
- Added a new test file to generate and verify model-specific tool snapshots.

Activity

The author has updated relevant documentation and README where necessary.
The author has added new tests to cover the introduced functionality.
The author has validated the changes on MacOS using npm run.
The author has validated the changes on Linux using npm run.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request adds snapshot tests for core tool definitions against different Gemini models. The tests are well-structured, using mocking to ensure deterministic output for platform-dependent tools. The snapshot files are organized clearly by model ID. I've found one issue regarding string escaping in the grep_search tool's snapshot, which seems inconsistent with its source definition. Otherwise, the changes look good and provide a solid foundation for tracking tool definition changes.

gemini-code-assist · 2026-02-10T03:57:50Z

packages/core/src/tools/definitions/__snapshots__/gemini-2.5-pro/grep_search.json

+    "type": "object",
+    "properties": {
+      "pattern": {
+        "description": "The regular expression (regex) pattern to search for within file contents (e.g., 'function\\s+myFunction', 'import\\s+\\{.*\\}\\s+from\\s+.*').",


There's an inconsistency in the escaping of the example regex in this snapshot. The source string in coreTools.ts for this description would produce JSON with unescaped curly braces ({}), but the snapshot has them escaped (\{ and \}).

This makes the snapshot incorrect as it doesn't match what the code would generate. The snapshot should be updated to reflect the actual output.

Note that applying this change will likely cause the test to fail. The root cause is likely in coreTools.ts, where the string isn't escaped correctly to produce the desired output. That file should be fixed, and then snapshots regenerated.

This issue is also present in the gemini-3-pro-preview snapshot.

"description": "The regular expression (regex) pattern to search for within file contents (e.g., 'function\s+myFunction', 'import\s+{.*}\s+from\s+.*').",

gemini-cli · 2026-02-10T04:17:54Z

Thank you for linking an issue! This pull request has been automatically reopened.

github-actions · 2026-02-10T04:20:26Z

Size Change: -2 B (0%)

Total Size: 23.9 MB

ℹ️ View Unchanged

Filename	Size	Change
`./bundle/gemini.js`	23.9 MB	-2 B (0%)
`./bundle/sandbox-macos-permissive-closed.sb`	1.03 kB	0 B
`./bundle/sandbox-macos-permissive-open.sb`	890 B	0 B
`./bundle/sandbox-macos-permissive-proxied.sb`	1.31 kB	0 B
`./bundle/sandbox-macos-restrictive-closed.sb`	3.29 kB	0 B
`./bundle/sandbox-macos-restrictive-open.sb`	3.36 kB	0 B
`./bundle/sandbox-macos-restrictive-proxied.sb`	3.56 kB	0 B

_{compressed-size-action}

gemini-cli · 2026-02-10T14:33:36Z

Hi there! Thank you for your contribution to Gemini CLI.

To improve our contribution process and better track changes, we now require all pull requests to be associated with an existing issue, as announced in our recent discussion and as detailed in our CONTRIBUTING.md.

This pull request is being closed because it is not currently linked to an issue. Once you have updated the description of this PR to link an issue (e.g., by adding Fixes #123 or Related to #123), it will be automatically reopened.

How to link an issue:
Add a keyword followed by the issue number (e.g., Fixes #123) in the description of your pull request. For more details on supported keywords and how linking works, please refer to the GitHub Documentation on linking pull requests to issues.

Thank you for your understanding and for being a part of our community!

gemini-cli · 2026-02-10T14:35:55Z

Thank you for linking an issue! This pull request has been automatically reopened.

mattKorwel

LGTM!

Co-authored-by: matt korwel <matt.korwel@gmail.com>

* Fix newline insertion bug in replace tool (google-gemini#18595) * fix(evals): update save_memory evals and simplify tool description (google-gemini#18610) * chore(evals): update validation_fidelity_pre_existing_errors to USUALLY_PASSES (google-gemini#18617) * fix: shorten tool call IDs and fix duplicate tool name in truncated output filenames (google-gemini#18600) * feat(cli): implement atomic writes and safety checks for trusted folders (google-gemini#18406) * Remove relative docs links (google-gemini#18650) * docs: add legacy snippets convention to GEMINI.md (google-gemini#18597) * fix(chore): Support linting for cjs (google-gemini#18639) Co-authored-by: Gal Zahavi <38544478+galz10@users.noreply.github.com> * feat: move shell efficiency guidelines to tool description (google-gemini#18614) * Added "" as default value, since getText() used to expect a string only and thus crashed when undefined... Fixes google-gemini#18076 (google-gemini#18099) * Allow @-includes outside of workspaces (with permission) (google-gemini#18470) * chore: make `ask_user` header description more clear (google-gemini#18657) * bug(core): Fix minor bug in migration logic. (google-gemini#18661) * Harded code assist converter. (google-gemini#18656) * refactor(core): model-dependent tool definitions (google-gemini#18563) * feat: enable plan mode experiment in settings (google-gemini#18636) * refactor: push isValidPath() into parsePastedPaths() (google-gemini#18664) * fix(cli): correct 'esc to cancel' position and restore duration display (google-gemini#18534) * feat(cli): add DevTools integration with gemini-cli-devtools (google-gemini#18648) * chore: remove unused exports and redundant hook files (google-gemini#18681) * Fix number of lines being reported in rewind confirmation dialog (google-gemini#18675) * feat(cli): disable folder trust in headless mode (google-gemini#18407) * Disallow unsafe type assertions (google-gemini#18688) * Change event type for release (google-gemini#18693) * feat: handle multiple dynamic context filenames in system prompt (google-gemini#18598) * Properly parse at-commands with narrow non-breaking spaces (google-gemini#18677) * refactor(core): centralize core tool definitions and support model-specific schemas (google-gemini#18662) * feat(core): Render memory hierarchically in context. (google-gemini#18350) * feat: Ctrl+O to expand paste placeholder (google-gemini#18103) * fix(cli): Improve header spacing (google-gemini#18531) * Feature/quota visibility 16795 (google-gemini#18203) * docs: remove TOC marker from Plan Mode header (google-gemini#18678) * Inline thinking bubbles with summary/full modes (google-gemini#18033) Co-authored-by: Jacob Richman <jacob314@gmail.com> * fix(ui): remove redundant newlines in Gemini messages (google-gemini#18538) * test(cli): fix AppContainer act() warnings and improve waitFor resilience (google-gemini#18676) * refactor(core): refine Security & System Integrity section in system prompt (google-gemini#18601) * Fix layout rounding. (google-gemini#18667) * docs(skills): enhance pr-creator safety and interactivity (google-gemini#18616) * test(core): remove hardcoded model from TestRig (google-gemini#18710) * feat(core): optimize sub-agents system prompt intro (google-gemini#18608) * feat(cli): update approval mode labels and shortcuts per latest UX spec (google-gemini#18698) * fix(plan): update persistent approval mode setting (google-gemini#18638) Co-authored-by: Sandy Tao <sandytao520@icloud.com> * fix: move toasts location to left side (google-gemini#18705) * feat(routing): restrict numerical routing to Gemini 3 family (google-gemini#18478) * fix(ide): fix ide nudge setting (google-gemini#18733) * fix(core): standardize tool formatting in system prompts (google-gemini#18615) * chore: consolidate to green in ask user dialog (google-gemini#18734) * feat: add `extensionsExplore` setting to enable extensions explore UI. (google-gemini#18686) * feat(cli): defer devtools startup and integrate with F12 (google-gemini#18695) * ui: update & subdue footer colors and animate progress indicator (google-gemini#18570) * test: add model-specific snapshots for coreTools (google-gemini#18707) Co-authored-by: matt korwel <matt.korwel@gmail.com> * ci: shard windows tests and fix event listener leaks (google-gemini#18670) * fix: allow `ask_user` tool in yolo mode (google-gemini#18541) * feat: redact disabled tools from system prompt (google-gemini#13597) (google-gemini#18613) * Update Gemini.md to use the curent year on creating new files (google-gemini#18460) * Code review cleanup for thinking display (google-gemini#18720) * fix(cli): hide scrollbars when in alternate buffer copy mode (google-gemini#18354) Co-authored-by: Jacob Richman <jacob314@gmail.com> * Fix issues with rip grep (google-gemini#18756) * fix(cli): fix history navigation regression after prompt autocomplete (google-gemini#18752) * chore: cleanup unused and add unlisted dependencies in packages/cli (google-gemini#18749) * Fix issue where Gemini CLI creates tests in a new file (google-gemini#18409) * feat(telemetry): Ensure experiment IDs are included in OpenTelemetry logs (google-gemini#18747) * feat(ux): added text wrapping capabilities to markdown tables (google-gemini#18240) Co-authored-by: jacob314 <jacob314@gmail.com> * Revert "fix(mcp): ensure MCP transport is closed to prevent memory leaks" (google-gemini#18771) * chore(release): bump version to 0.30.0-nightly.20260210.a2174751d (google-gemini#18772) * chore: cleanup unused and add unlisted dependencies in packages/core (google-gemini#18762) * chore(core): update activate_skill prompt verbiage to be more direct (google-gemini#18605) * Add autoconfigure memory usage setting to the dialog (google-gemini#18510) Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * fix(core): prevent race condition in policy persistence (google-gemini#18506) Co-authored-by: Allen Hutchison <adh@google.com> * fix(evals): prevent false positive in hierarchical memory test (google-gemini#18777) * test(evals): mark all `save_memory` evals as `USUALLY_PASSES` due to unreliability (google-gemini#18786) * feat(cli): add setting to hide shortcuts hint UI (google-gemini#18562) * feat(core): formalize 5-phase sequential planning workflow (google-gemini#18759) * Introduce limits for search results. (google-gemini#18767) --------- Co-authored-by: Andrew Garrett <andrewgarrett@google.com> Co-authored-by: N. Taylor Mullen <ntaylormullen@google.com> Co-authored-by: Sandy Tao <sandytao520@icloud.com> Co-authored-by: Gal Zahavi <38544478+galz10@users.noreply.github.com> Co-authored-by: christine betts <chrstn@uw.edu> Co-authored-by: Aswin Ashok <aswwwin@google.com> Co-authored-by: Abhijith V Ashok <abhi2349jith@gmail.com> Co-authored-by: Tommaso Sciortino <sciortino@gmail.com> Co-authored-by: Jack Wotherspoon <jackwoth@google.com> Co-authored-by: joshualitt <joshualitt@google.com> Co-authored-by: Jacob Richman <jacob314@gmail.com> Co-authored-by: Aishanee Shah <aishaneeshah@gmail.com> Co-authored-by: Jerop Kipruto <jerop@google.com> Co-authored-by: Adib234 <30782825+Adib234@users.noreply.github.com> Co-authored-by: Christian Gunderman <gundermanc@gmail.com> Co-authored-by: g-samroberts <158088236+g-samroberts@users.noreply.github.com> Co-authored-by: Spencer <spencertang@google.com> Co-authored-by: Dmitry Lyalin <dmitry.lyalin@lyalin.com> Co-authored-by: matt korwel <matt.korwel@gmail.com> Co-authored-by: Shreya Keshive <shreyakeshive@google.com> Co-authored-by: Sri Pasumarthi <111310667+sripasg@users.noreply.github.com> Co-authored-by: Keith Guerin <keithguerin@gmail.com> Co-authored-by: Sehoon Shon <sshon@google.com> Co-authored-by: Adam Weidman <65992621+adamfweidman@users.noreply.github.com> Co-authored-by: Kevin Ramdass <ramdass.kevin@gmail.com> Co-authored-by: Dev Randalpura <devrandalpura@google.com> Co-authored-by: gemini-cli-robot <gemini-cli-robot@google.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Brad Dux <959674+braddux@users.noreply.github.com> Co-authored-by: Allen Hutchison <adh@google.com> Co-authored-by: Abhijit Balaji <abhijitbalaji@google.com>

aishaneeshah added 3 commits February 10, 2026 03:23

test: add model-specific snapshots for coreTools

44a09cf

test: reorganize coreTools snapshots into directory structure by model

20dd832

test: mock os.platform for deterministic coreTools snapshots

6c78339

aishaneeshah requested a review from a team as a code owner February 10, 2026 03:55

gemini-cli bot closed this Feb 10, 2026

gemini-code-assist bot reviewed Feb 10, 2026

View reviewed changes

gemini-cli bot reopened this Feb 10, 2026

gemini-cli bot added area/agent Issues related to Core Agent, Tools, Memory, Sub-Agents, Hooks, Agent Quality 🔒 maintainer only ⛔ Do not contribute. Internal roadmap item. labels Feb 10, 2026

aishaneeshah added 3 commits February 10, 2026 13:57

test: multi-line required arrays in coreTools snapshots

88d5f10

test: update coreTools snapshots with multi-line required arrays

c9c71ef

test: migrate to standard Vitest snapshots for coreTools

ce8721b

gemini-cli bot closed this Feb 10, 2026

gemini-cli bot reopened this Feb 10, 2026

aishaneeshah mentioned this pull request Feb 10, 2026

Tool Improvements: Ensure Tool changes are avaialble per model #17958

Closed

aishaneeshah enabled auto-merge February 10, 2026 15:08

mattKorwel approved these changes Feb 10, 2026

View reviewed changes

mattKorwel and others added 2 commits February 10, 2026 10:49

Merge branch 'main' into feat/core-tools-model-snapshots

1aed0c7

Merge branch 'main' into feat/core-tools-model-snapshots

2d7873b

aishaneeshah added this pull request to the merge queue Feb 10, 2026

github-merge-queue bot pushed a commit that referenced this pull request Feb 10, 2026

test: add model-specific snapshots for coreTools (#18707)

e41ab37

Co-authored-by: matt korwel <matt.korwel@gmail.com>

Merged via the queue into main with commit 262138c Feb 10, 2026
26 checks passed

aishaneeshah deleted the feat/core-tools-model-snapshots branch February 10, 2026 19:02

This was referenced Feb 18, 2026

Changelog for v0.29.0 #19361

Merged

Changelog for v0.30.0-preview.5 #20107

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

test: add model-specific snapshots for coreTools#18707

test: add model-specific snapshots for coreTools#18707
aishaneeshah merged 8 commits intomainfrom
feat/core-tools-model-snapshots

aishaneeshah commented Feb 10, 2026 •

edited

Loading

Uh oh!

gemini-cli bot commented Feb 10, 2026

Uh oh!

gemini-cli bot commented Feb 10, 2026

Uh oh!

gemini-code-assist bot commented Feb 10, 2026

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

gemini-code-assist bot Feb 10, 2026

Uh oh!

gemini-cli bot commented Feb 10, 2026

Uh oh!

github-actions bot commented Feb 10, 2026 •

edited

Loading

Uh oh!

gemini-cli bot commented Feb 10, 2026

Uh oh!

gemini-cli bot commented Feb 10, 2026

Uh oh!

mattKorwel left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Comments

Conversation

aishaneeshah commented Feb 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Details

Related Issues

How to Validate

Pre-Merge Checklist

Uh oh!

gemini-cli bot commented Feb 10, 2026

Uh oh!

gemini-cli bot commented Feb 10, 2026

Uh oh!

gemini-code-assist bot commented Feb 10, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist bot Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-cli bot commented Feb 10, 2026

Uh oh!

github-actions bot commented Feb 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gemini-cli bot commented Feb 10, 2026

Uh oh!

gemini-cli bot commented Feb 10, 2026

Uh oh!

mattKorwel left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

aishaneeshah commented Feb 10, 2026 •

edited

Loading

github-actions bot commented Feb 10, 2026 •

edited

Loading