fix: Windows Specific Agent Quality & System Prompt

[scidomino]

Summary

Windows Specific Agent Quality & System Prompt

Related Issues

Fixes #17137

How to Validate

Run on windows (and non windows) and ask the agent if it sees os appropriate text.

Pre-Merge Checklist

• Updated relevant documentation and README (if needed)
• Added/updated tests (if needed)
• Noted breaking changes (if any)
• Validated on required platforms/methods:
  • MacOS
    • npm run
    • npx
    • Docker
    • Podman
    • Seatbelt
  • Windows
    • npm run
    • npx
    • Docker
  • Linux
    • npm run
    • npx
    • Docker

[gemini-code-assist]

Summary of Changes

Hello @scidomino, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the agent's ability to operate effectively across different operating systems by introducing platform-aware system prompts. It ensures that agents receive and provide instructions tailored to the specific OS environment, improving the accuracy and usability of shell commands for tasks like file listing and temporary file inspection. Additionally, it refines the guidance for running background processes, promoting a more robust and consistent approach.

Highlights

• Platform-Specific System Prompts: The system prompts for agents, specifically the Codebase Investigator, now dynamically adapt to the operating system. On Windows, it suggests dir /s or Get-ChildItem -Recurse for listing files, while on other platforms like Linux, it suggests ls -R.
• Updated Shell Efficiency Guidelines: The system prompt's guidelines for inspecting temporary files have been updated to provide platform-specific command examples. For Windows, it now suggests type, findstr, Get-Content, or Select-String, whereas for non-Windows, it continues to suggest grep, tail, or head.
• Refined Background Process Instructions: Instructions within the system prompt for running background processes have been clarified. Agents are now guided to use the is_background parameter for the run_shell_command tool instead of the & operator, improving consistency and reliability.
• Enhanced Test Coverage: New tests have been added to verify the platform-specific behavior of system prompts and the updated instructions for background processes, ensuring correct agent behavior across different operating systems.

Changelog

• packages/core/src/agents/codebase-investigator.test.ts
  • Added vi and afterEach imports for testing utilities.
  • Introduced an afterEach hook to ensure global mocks are reset after each test.
  • Implemented a mockPlatform helper function to facilitate platform-specific testing by stubbing process.platform.
  • Modified existing tests to instantiate CodebaseInvestigatorAgent within each test to ensure a clean state after platform mocking.
  • Added new tests to verify that the system prompt includes Windows-specific list commands (dir /s or Get-ChildItem -Recurse) when process.platform is 'win32'.
  • Added new tests to verify that the system prompt includes generic list commands (ls -R) when process.platform is non-Windows (e.g., 'linux').
• packages/core/src/agents/codebase-investigator.ts
  • Introduced a listCommand variable that dynamically selects between Windows-specific (dir /s or Get-ChildItem -Recurse) and generic (ls -R) file listing commands based on process.platform.
  • Updated the ExplorationTrace example in the system prompt to use the listCommand variable, making it platform-aware.
• packages/core/src/core/snapshots/prompts.test.ts.snap
  • Updated snapshot tests to reflect changes in the system prompt, including platform-specific command examples.
  • Removed the phrase "(or platform equivalents)" from the temporary file inspection guideline.
  • Updated the "Background Processes" guideline to refer to the is_background parameter instead of via &.
• packages/core/src/core/prompts.test.ts
  • Added afterEach import and hook to unstub global mocks.
  • Modified beforeEach to stub process.platform to 'linux' by default for consistent snapshot testing.
  • Added a new describe block for "Platform-specific and Background Process instructions" tests.
  • Introduced a mockPlatform helper function for platform-specific testing.
  • Added tests to verify Windows-specific shell efficiency commands (type, findstr, Get-Content, Select-String) in the system prompt.
  • Added tests to verify generic shell efficiency commands (grep, tail, head) in the system prompt for non-Windows platforms.
  • Added a test to confirm the updated background process instruction using the is_background parameter.
• packages/core/src/prompts/snippets.ts
  • Modified the shellEfficiencyGuidelines function to dynamically set inspectExample based on process.platform (Windows vs. non-Windows).
  • Updated the temporary file inspection guideline to use the inspectExample variable, providing platform-specific command examples.
  • Updated the toolUsageInteractive function to change the "Background Processes" instruction to recommend using the is_background parameter instead of via &.

Activity

• The pull request was opened by scidomino with the title 'fix: Windows Specific Agent Quality & System Prompt'.
• The author provided a summary indicating the focus on Windows-specific agent quality and system prompts.
• The PR addresses and fixes issue Windows Specific Agent Quality & System Prompt #17137, as noted in the description.
• Validation instructions are provided, suggesting testing on both Windows and non-Windows environments to confirm OS-appropriate text.
• A pre-merge checklist is included, outlining necessary steps for documentation, testing, breaking changes, and validation across MacOS, Windows, and Linux using various methods (npm run, npx, Docker, Podman, Seatbelt).

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces two key improvements to the system prompts to enhance agent quality, particularly on Windows. Firstly, it makes prompts platform-aware by providing Windows-specific command examples (like dir /s) alongside their Unix-like counterparts (ls -R). Secondly, it updates the instructions for running background processes, guiding the agent to use the more robust is_background parameter instead of the shell's & operator. The changes are well-implemented and thoroughly tested, including new unit tests that mock the operating system platform to ensure correctness. The code quality is high, and I have no further recommendations.

[github-actions]

Size Change: +246 B (0%)

Total Size: 23.7 MB

ℹ️ View Unchanged

Filename	Size	Change
./bundle/gemini.js	23.7 MB	+246 B (0%)
./bundle/sandbox-macos-permissive-closed.sb	1.03 kB	0 B
./bundle/sandbox-macos-permissive-open.sb	890 B	0 B
./bundle/sandbox-macos-permissive-proxied.sb	1.31 kB	0 B
./bundle/sandbox-macos-restrictive-closed.sb	3.29 kB	0 B
./bundle/sandbox-macos-restrictive-open.sb	3.36 kB	0 B
./bundle/sandbox-macos-restrictive-proxied.sb	3.56 kB	0 B

_{compressed-size-action}

[jacob314]

Fyi @galz10 was it an error that we told the agent to use & instead of is_background?

[jacob314]