feat(core): experimental in-progress steering hints

[joshualitt]

This is a rebase / refactor of:
#18783

[gemini-code-assist]

Summary of Changes

Hello @joshualitt, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces an experimental capability for real-time model steering, allowing users to provide dynamic guidance to the AI agent during complex tasks. It integrates a new lightweight model for efficient hint processing and acknowledgement, updates the model's core instructions to properly handle these hints, and provides a robust testing framework to ensure the feature's reliability. The changes aim to enhance user control and interaction fluidity with the AI agent.

Highlights

• Experimental In-Progress Steering Hints: Introduced an experimental feature allowing users to provide real-time 'steering hints' to guide the model's ongoing execution. These hints are processed to influence the model's plan without interrupting its flow.
• New flash-lite-helper Model Configuration: Added a new model configuration specifically for lightweight, low-latency tasks like generating acknowledgements for user hints, leveraging the gemini-2.5-flash-lite model.
• Updated System Prompts: Modified core system prompts to include explicit instructions for the model on how to interpret and respond to user steering hints, emphasizing minimal, scope-preserving course corrections.
• Enhanced UI for Hints: Implemented new UI components and updated existing ones to display user-provided hints distinctly in the chat history and manage hint-related input and state.
• Comprehensive Testing Utilities: Developed new integration testing utilities (AppRig, MockShellExecutionService) to simulate user interactions and model responses, specifically enabling robust testing of the model steering functionality.

Changelog

• docs/get-started/configuration.md
  • Added 'flash-lite-helper' model configuration to the documentation.
• packages/cli/src/integration-tests/modelSteering.test.tsx
  • Added new integration test for model steering during tool execution.
• packages/cli/src/test-utils/AppRig.test.tsx
  • Added a basic test for the AppRig utility.
• packages/cli/src/test-utils/AppRig.tsx
  • Added the AppRig class for UI integration testing, including methods for injecting user hints and resolving tool confirmations.
• packages/cli/src/test-utils/MockShellExecutionService.ts
  • Added MockShellExecutionService to simulate shell command execution in tests.
• packages/cli/src/test-utils/fixtures/simple.responses
  • Added a fixture for simple model responses.
• packages/cli/src/test-utils/fixtures/steering.responses
  • Added a fixture for model steering responses.
• packages/cli/src/test-utils/render.tsx
  • Updated renderWithProviders to include SessionStatsProvider and hint-related UI state and actions.
  • Added 'hintMode' and 'hintBuffer' to the base mock UI state.
  • Added hint-related actions to mock UIActions.
• packages/cli/src/ui/AppContainer.tsx
  • Implemented logic to consume pending user hints and generate steering acknowledgements.
  • Modified handleFinalSubmit to process user input as a hint if the agent is running and the input is not a slash command.
  • Added consumePendingHints to the useGeminiStream dependency array.
  • Added hintMode and hintBuffer to the UI state context provider.
  • Added placeholder functions for hint-related actions to the UIActions context provider.
• packages/cli/src/ui/components/DialogManager.tsx
  • Formatted import statements.
• packages/cli/src/ui/components/Footer.tsx
  • Updated logic for displaying the debug profiler based on development mode and settings.
• packages/cli/src/ui/components/Header.test.tsx
  • Added 'hintMode' property to the background theme in tests.
• packages/cli/src/ui/components/HistoryItemDisplay.test.tsx
  • Added a test case for rendering the new HintMessage component.
• packages/cli/src/ui/components/HistoryItemDisplay.tsx
  • Added HintMessage component to display history items of type 'hint'.
  • Added marginBottom prop to InfoMessage component.
• packages/cli/src/ui/components/InputPrompt.tsx
  • Updated useShellHistory hook to pass the config storage.
• packages/cli/src/ui/components/messages/HintMessage.tsx
  • Added new HintMessage component for displaying user steering hints with a distinct style.
• packages/cli/src/ui/components/messages/InfoMessage.tsx
  • Added marginBottom prop to the InfoMessage component for spacing control.
• packages/cli/src/ui/contexts/UIActionsContext.tsx
  • Added new actions (onHintInput, onHintBackspace, onHintClear, onHintSubmit) to the UIActions interface.
• packages/cli/src/ui/contexts/UIStateContext.tsx
  • Added 'hintMode' and 'hintBuffer' properties to the UIState interface.
• packages/cli/src/ui/hooks/useGeminiStream.test.tsx
  • Updated MockedGeminiClientClass to include a mock for generateContent.
  • Updated useSessionStats mock to use importOriginal.
  • Added a test case to verify steering hint injection for continuation prompts and acknowledgement generation.
• packages/cli/src/ui/hooks/useGeminiStream.ts
  • Imported buildUserSteeringHintPrompt and generateSteeringAckMessage from @google/gemini-cli-core.
  • Added consumeUserHint parameter to useGeminiStream.
  • Implemented logic to inject user hints into model prompts and generate acknowledgements using flashLiteHelper.
• packages/cli/src/ui/hooks/useShellHistory.ts
  • Added error handling and an isMounted check to the useShellHistory hook to prevent state updates on unmounted components.
• packages/cli/src/ui/themes/no-color.ts
  • Added 'hintMode' to the background theme for no-color settings.
• packages/cli/src/ui/themes/semantic-tokens.ts
  • Added 'hintMode' property to the SemanticColors interface.
  • Defined 'hintMode' colors for both light and dark semantic themes.
• packages/cli/src/ui/themes/theme.ts
  • Added 'hintMode' color to the Theme class based on theme type.
• packages/cli/src/ui/types.ts
  • Added new type HistoryItemHint for user steering hints.
  • Added HINT to the MessageType enum.
• packages/core/src/agents/local-executor.ts
  • Modified LocalAgentExecutor to incorporate initial and new user hints into the model's content messages.
• packages/core/src/agents/subagent-tool.ts
  • Imported formatUserHintsForModel.
  • Modified shouldConfirmExecute and execute methods to pass user hints to sub-agent invocations.
  • Added withUserHints private method to format and inject user hints into agent arguments.
• packages/core/src/config/config.test.ts
  • Added tests for addUserHint, getUserHints, getUserHintsAfter, getLatestHintIndex, and getLastUserHintAt methods in the Config class.
• packages/core/src/config/config.ts
  • Added userHints array and methods (addUserHint, getUserHints, getUserHintsAfter, getLatestHintIndex, getLastUserHintAt) to manage user steering hints.
• packages/core/src/config/defaultModelConfigs.ts
  • Added 'flash-lite-helper' model configuration to default model configs.
• packages/core/src/core/snapshots/prompts.test.ts.snap
  • Updated snapshots to include the new 'User Hints' mandate in the system prompt.
• packages/core/src/core/prompts.test.ts
  • Added assertions to verify the inclusion of the 'User Hints' mandate in the generated system prompts.
• packages/core/src/index.ts
  • Exported flashLiteHelper.ts utilities.
• packages/core/src/prompts/snippets.legacy.ts
  • Added 'User Hints' mandate to the legacy system prompt snippet.
• packages/core/src/prompts/snippets.ts
  • Added 'User Hints' mandate to the core system prompt snippet.
• packages/core/src/services/test-data/resolved-aliases-retry.golden.json
  • Updated golden file to include 'flash-lite-helper' model configuration.
• packages/core/src/services/test-data/resolved-aliases.golden.json
  • Updated golden file to include 'flash-lite-helper' model configuration.
• packages/core/src/utils/flashLiteHelper.test.ts
  • Added tests for truncateFlashLiteInput, generateFlashLiteText, and generateSteeringAckMessage functions.
• packages/core/src/utils/flashLiteHelper.ts
  • Added utility functions for Flash Lite model interactions, including input truncation, prompt building for steering hints, and acknowledgement message generation.
• schemas/settings.schema.json
  • Updated the settings schema to include the 'flash-lite-helper' model configuration.

Activity

• This pull request is a rebase and refactor of a previous pull request (feat: experimental in-progress steering hints #18783), indicating ongoing development and refinement of the steering hints feature.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces an experimental feature for in-progress steering hints, allowing users to guide the model's execution while a task is running. The implementation is comprehensive, touching the core logic, UI, and agent execution. Key changes include updating the Config class to manage hints, modifying the system prompt to instruct the model on handling these hints, and updating the UI to capture user input as hints during an active task. A new flash-lite-helper utility is introduced for generating quick user acknowledgements, which is a nice UX touch. The feature is also integrated into local and remote agent execution. The addition of a new AppRig test utility and corresponding integration tests demonstrates a high level of quality and thought. The code is well-structured and I have no high or critical severity feedback.

[gemini-cli]

Hi there! Thank you for your contribution to Gemini CLI.

To improve our contribution process and better track changes, we now require all pull requests to be associated with an existing issue, as announced in our recent discussion and as detailed in our CONTRIBUTING.md.

This pull request is being closed because it is not currently linked to an issue. Once you have updated the description of this PR to link an issue (e.g., by adding Fixes #123 or Related to #123), it will be automatically reopened.

How to link an issue:
Add a keyword followed by the issue number (e.g., Fixes #123) in the description of your pull request. For more details on supported keywords and how linking works, please refer to the GitHub Documentation on linking pull requests to issues.

Thank you for your understanding and for being a part of our community!