Skip to content

feat: experimental in-progress steering hints#18783

Draft
LyalinDotCom wants to merge 1 commit intomainfrom
feature-hint-agent
Draft

feat: experimental in-progress steering hints#18783
LyalinDotCom wants to merge 1 commit intomainfrom
feature-hint-agent

Conversation

@LyalinDotCom
Copy link
Collaborator

Summary

This PR adds an experimental steering-hint flow so users can steer the agent while a turn is already in progress.

Refs #18782

What changed

  • Keeps normal input usable while the agent is running.
  • Routes non-slash input during active execution into steering hints.
  • Injects a hidden steering prompt into continuation/follow-up turns.
  • Flushes pending hints after a turn if immediate continuation injection was missed.
  • Removes warning-like hint status styling and uses neutral history feedback.
  • Adds a reusable core flash-lite helper utility and a dedicated flash-lite-helper model config alias.
  • Generates short steering acknowledgement text via flash-lite (with truncation, timeout, and fallback).

Experimental scope

This is intentionally experimental so contributors can try it end-to-end and provide feedback before we harden behavior and UX.

Validation

  • npm run -w @google/gemini-cli-core typecheck
  • npm run -w @google/gemini-cli typecheck
  • npm run -w @google/gemini-cli-core test -- src/services/modelConfig.golden.test.ts src/utils/flashLiteHelper.test.ts
  • npm run -w @google/gemini-cli test -- src/ui/hooks/useGeminiStream.test.tsx src/ui/components/HistoryItemDisplay.test.tsx
  • targeted lint for touched core/cli files

@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello @LyalinDotCom, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces an experimental feature that significantly enhances user interaction by allowing real-time 'steering hints' to be provided to the agent during active execution. This enables users to guide or correct the agent's behavior mid-process, making the interaction more dynamic and responsive. The implementation includes updates to input handling, prompt injection mechanisms, and a new lightweight model for generating immediate user acknowledgments.

Highlights

  • Real-time Steering Hints: Introduced an experimental feature allowing users to provide input as 'steering hints' while the agent is actively running, enabling dynamic course correction.
  • Dynamic Prompt Injection: Implemented logic to automatically inject these user-provided hints into subsequent agent prompts, ensuring the agent can adapt its ongoing plan based on real-time feedback.
  • Flash-Lite Acknowledgment System: Added a new flash-lite helper utility and model configuration to generate concise, real-time acknowledgments for steering hints, providing immediate user feedback without interrupting the agent's flow.
  • Enhanced UI/UX for Hints: The user interface now supports a dedicated 'hint mode' with appropriate styling and history display for steering hints, clearly distinguishing them from regular commands.
  • Configurable Hint Management: The core configuration now includes mechanisms to store, peek, and consume user hints, along with tracking their timestamps for better management and processing.
Changelog
  • packages/cli/src/test-utils/render.tsx
    • Updated mock UI state to include hintMode and hintBuffer.
    • Added mock hint-related actions to mockUIActions.
  • packages/cli/src/ui/AppContainer.tsx
    • Imported steering hint utility functions.
    • Implemented consumePendingHints and getUserHint for managing hints from the configuration.
    • Integrated getUserHint into the useGeminiStream hook.
    • Added handleHintSubmit to process and display user hints.
    • Modified handleFinalSubmit to route non-slash input to handleHintSubmit when the agent is active.
    • Added a useEffect hook to process pending hints, generate acknowledgments, and submit steering prompts when the agent is idle.
    • Initialized hintMode and hintBuffer in the UI state and actions.
  • packages/cli/src/ui/components/Footer.tsx
    • Updated the condition for displaying the debug profiler.
  • packages/cli/src/ui/components/Header.test.tsx
    • Added hintMode property to the mock background theme.
  • packages/cli/src/ui/components/HistoryItemDisplay.test.tsx
    • Added a test case for rendering HintMessage type.
  • packages/cli/src/ui/components/HistoryItemDisplay.tsx
    • Imported and conditionally rendered the HintMessage component.
    • Passed the marginBottom prop to InfoMessage.
  • packages/cli/src/ui/components/messages/InfoMessage.tsx
    • Added marginBottom prop to control vertical spacing.
  • packages/cli/src/ui/contexts/UIActionsContext.tsx
    • Extended the UIActions interface with new hint input, backspace, clear, and submit actions.
  • packages/cli/src/ui/contexts/UIStateContext.tsx
    • Added hintMode and hintBuffer properties to the UIState interface.
  • packages/cli/src/ui/hooks/useGeminiStream.test.tsx
    • Mocked generateContent for GeminiClient to simulate steering acknowledgment.
    • Added a test to verify that steering hint prompts are injected for continuation turns and acknowledgment messages are added to the history.
  • packages/cli/src/ui/hooks/useGeminiStream.ts
    • Added utility functions buildUserSteeringHintPrompt and generateSteeringAckMessage, along with related constants.
    • Modified the useGeminiStream hook to accept getUserHint and inject hints into tool responses.
    • Added getUserHint to the dependencies of submitToolResponses.
  • packages/cli/src/ui/themes/no-color.ts
    • Added hintMode to the background colors in the no-color theme.
  • packages/cli/src/ui/themes/semantic-tokens.ts
    • Added hintMode to the SemanticColors interface and defined its values for light and dark themes.
  • packages/cli/src/ui/themes/theme.ts
    • Included hintMode in the theme background definitions for both the Theme class and createCustomTheme function.
  • packages/cli/src/ui/types.ts
    • Added marginBottom to the HistoryItemInfo type.
    • Defined a new HistoryItemHint type and added it to the HistoryItemWithoutId union type.
  • packages/core/src/agents/local-executor.ts
    • Modified prompt construction to include user hints when present.
  • packages/core/src/agents/subagent-tool.ts
    • Implemented withUserHints to pass hints to sub-agent invocations.
    • Modified shouldConfirmExecute and execute to utilize withUserHints.
  • packages/core/src/config/config.test.ts
    • Added unit tests for user hint storage, retrieval, and consumption methods.
  • packages/core/src/config/config.ts
    • Added private state (pendingUserHints, lastUserHintAt) and public methods (addUserHint, peekUserHints, consumeUserHints, getLastUserHintAt) for managing user hints.
  • packages/core/src/config/defaultModelConfigs.ts
    • Added a new flash-lite-helper model configuration with specific generation settings.
  • packages/core/src/core/snapshots/prompts.test.ts.snap
    • Updated system prompt snapshots to include the new 'User Hints' mandate.
  • packages/core/src/core/prompts.test.ts
    • Updated prompt tests to assert the presence of the 'User Hints' mandate in system prompts.
  • packages/core/src/index.ts
    • Exported the new flashLiteHelper.js utility.
  • packages/core/src/prompts/snippets.legacy.ts
    • Included the 'User Hints' mandate in legacy core mandates.
  • packages/core/src/prompts/snippets.ts
    • Included the 'User Hints' mandate in core mandates.
  • packages/core/src/services/test-data/resolved-aliases-retry.golden.json
    • Updated golden test data to include the flash-lite-helper model configuration.
  • packages/core/src/services/test-data/resolved-aliases.golden.json
    • Updated golden test data to include the flash-lite-helper model configuration.
  • packages/core/src/utils/flashLiteHelper.test.ts
    • Added unit tests for truncateFlashLiteInput and generateFlashLiteText functions.
  • packages/core/src/utils/flashLiteHelper.ts
    • Created a new utility file implementing truncateFlashLiteInput and generateFlashLiteText for concise text generation with truncation and fallback.
Activity
  • This pull request introduces an experimental feature, indicating it's a new development phase for testing and gathering feedback from contributors before finalizing behavior and user experience.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces an experimental feature for in-progress steering hints, allowing users to guide the agent during an active turn. The implementation is comprehensive, covering UI, state management, and core agent logic. However, I've identified a critical security vulnerability related to prompt injection. The user-provided hint text is not sanitized before being embedded into LLM prompts, which could allow a malicious user to manipulate the agent's behavior. My review includes a specific code suggestion to address this vulnerability by sanitizing the input.

Comment on lines +2492 to +2497
const trimmed = hint.trim();
if (trimmed.length === 0) {
return;
}
this.pendingUserHints.push(trimmed);
this.lastUserHintAt = Date.now();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

The user-provided hint is added to the list of pending hints without proper sanitization, which could lead to a prompt injection vulnerability. The hint text is later embedded directly into prompts for the LLM. A malicious user could craft a hint containing special characters or directives (e.g., XML-like tags) to manipulate the prompt and potentially alter the agent's behavior.

According to the general rules, to prevent prompt injection, any user-provided context should be sanitized by escaping characters like < and >. The sanitization should be applied here in addUserHint to ensure all consumers of the hints receive the safe version.

Suggested change
const trimmed = hint.trim();
if (trimmed.length === 0) {
return;
}
this.pendingUserHints.push(trimmed);
this.lastUserHintAt = Date.now();
const sanitized = hint.trim().replace(/</g, '&lt;').replace(/>/g, '&gt;');
if (sanitized.length === 0) {
return;
}
this.pendingUserHints.push(sanitized);
this.lastUserHintAt = Date.now();
References
  1. To prevent prompt injection, sanitize any additional context from hooks by escaping HTML-like tag characters such as < and >. Although steering hints are not from hooks, they are user-provided context and the same sanitization principle applies.

@gemini-cli gemini-cli bot added area/agent Issues related to Core Agent, Tools, Memory, Sub-Agents, Hooks, Agent Quality 🔒 maintainer only ⛔ Do not contribute. Internal roadmap item. labels Feb 11, 2026
joshualitt added a commit that referenced this pull request Feb 12, 2026
joshualitt added a commit that referenced this pull request Feb 13, 2026
joshualitt added a commit that referenced this pull request Feb 13, 2026
Roll-up PR here:
#18973

This is a rebase / refactor of:
#18783

Part of a series to address issue #18782
joshualitt added a commit that referenced this pull request Feb 13, 2026
Roll-up PR here:
#18973

This is a rebase / refactor of:
#18783

Part of a series to address issue #18782
joshualitt added a commit that referenced this pull request Feb 13, 2026
Roll-up PR here:
#18973

This is a rebase / refactor of:
#18783

Part of a series to address issue #18782
joshualitt added a commit that referenced this pull request Feb 13, 2026
Roll-up PR here:
#18973

This is a rebase / refactor of:
#18783

Part of a series to address issue #18782
joshualitt added a commit that referenced this pull request Feb 13, 2026
Roll-up PR here:
#18973

This is a rebase / refactor of:
#18783

Part of a series to address issue #18782
joshualitt added a commit that referenced this pull request Feb 17, 2026
Roll-up PR here:
#18973

This is a rebase / refactor of:
#18783

Part of a series to address issue #18782
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/agent Issues related to Core Agent, Tools, Memory, Sub-Agents, Hooks, Agent Quality 🔒 maintainer only ⛔ Do not contribute. Internal roadmap item.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants