Send accepted/removed lines with ACCEPT_FILE telemetry.

[gundermanc]

Summary

Updates the Code Assist telemetry to send the number of lines accepted or removed with each edit.

How to Validate

Login with a Google account and then ask Gemini CLI to make a change to a file.

Pre-Merge Checklist

• Updated relevant documentation and README (if needed)
• Added/updated tests (if needed)
• Noted breaking changes (if any)
• Validated on required platforms/methods:
  • MacOS
    • npm run
    • npx
    • Docker
    • Podman
    • Seatbelt
  • Windows
    • npm run
    • npx
    • Docker
  • Linux
    • npm run
    • npx
    • Docker

[gemini-code-assist]

Summary of Changes

Hello @gundermanc, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces more granular telemetry data for accepted file edits. By capturing the exact number of lines added and removed by the model, we can gain a clearer understanding of how code assistance features are impacting user workflows and the codebase. This enhancement improves the quality of collected metrics without altering the core functionality of the code assistance tools.

Highlights

• Enhanced Telemetry for Accepted Files: The system now collects and sends the number of lines added and removed by the model when a file edit is accepted, providing more detailed insights into the impact of code assistance.
• Telemetry Data Calculation: The summarizeToolCalls function has been updated to compute acceptedLines and removedLines from the tool call's response display for successful edit operations.
• API and Type Definition Updates: The ConversationInteraction interface and the createConversationInteraction function were modified to accommodate the new acceptedLines and removedLines fields.
• Refined Test Coverage: Existing telemetry tests were updated to assert the presence and correctness of the new line count data for ACCEPT_FILE interactions, and tests for non-edit tool interactions were clarified.

Changelog

• packages/core/src/code_assist/telemetry.test.ts
  • Added response.resultDisplay.diffStat to mock CompletedToolCall objects to simulate line changes.
  • Included acceptedLines, removedLines, and initiationMethod in the expected recordConversationInteraction call for ACCEPT_FILE.
  • Updated the description of a test case to clarify that interaction should not be recorded for non-edit accepted tools.
  • Removed the expectation for UNKNOWN interaction recording for non-edit accepted tools.
• packages/core/src/code_assist/telemetry.ts
  • Imported computeModelAddedAndRemovedLines and getFileDiffFromResultDisplay to assist with diff calculation.
  • Initialized acceptedLines and removedLines variables within summarizeToolCalls to aggregate line counts.
  • Implemented logic to calculate and accumulate acceptedLines and removedLines from successful edit tool calls.
  • Modified the condition for creating a ConversationInteraction to ensure it's only for edit actions when line counts are relevant.
  • Updated the createConversationInteraction function signature and its return object to accept and include acceptedLines and removedLines.
• packages/core/src/code_assist/types.ts
  • Added an optional removedLines property of type string to the ConversationInteraction interface.
• packages/core/src/utils/fileDiffUtils.ts
  • Imported the DiffStat type for more precise type annotations.
  • Simplified the getFileDiffFromResultDisplay function's internal logic for checking diffStat presence.
  • Updated the type of the stats parameter in computeModelAddedAndRemovedLines to DiffStat | undefined.

Activity

• No human activity has been recorded on this pull request yet, as indicated by the empty description sections.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

The pull request successfully adds acceptedLines and removedLines to the ACCEPT_FILE telemetry, which is a valuable improvement for tracking the impact of AI-generated edits. However, there are two significant issues that need to be addressed, both related to ensuring the accuracy and completeness of telemetry data as per established guidelines: a regression in the telemetry logic that filters out non-edit tool calls, and a mismatch in the tests where an unsupported property is expected. I have provided feedback and suggestions to resolve these issues, emphasizing the importance of semantically accurate telemetry keys.

[gemini-code-assist]

The addition of isEdit to the return condition causes the function to return undefined for non-edit tool calls (e.g., read_file), which were previously recorded as UNKNOWN. This change contradicts the comment on line 115 ("Treat file edits as ACCEPT_FILE and everything else as unknown") and results in a loss of telemetry coverage for general tool interactions. The suggested change restores the previous behavior while maintaining the new line count functionality for edits.

  return traceId && acceptedToolCalls / toolCalls.length >= 1
    ? createConversationInteraction(
        traceId,
        actionStatus || ActionStatus.ACTION_STATUS_NO_ERROR,
        isEdit
          ? ConversationInteractionInteraction.ACCEPT_FILE
          : ConversationInteractionInteraction.UNKNOWN,
        isEdit ? String(acceptedLines) : undefined,
        isEdit ? String(removedLines) : undefined,
      )
    : undefined;

References

1. When logging events, ensure that the keys used for telemetry data accurately reflect the semantic meaning of the data being logged to avoid misinterpretation. Introduce new, more specific keys when existing ones are semantically confusing in a given context.

[github-actions]

Size Change: +586 B (0%)

Total Size: 25.2 MB

ℹ️ View Unchanged

Filename	Size	Change
./bundle/gemini.js	25.1 MB	+586 B (0%)
./bundle/sandbox-macos-permissive-open.sb	890 B	0 B
./bundle/sandbox-macos-permissive-proxied.sb	1.31 kB	0 B
./bundle/sandbox-macos-restrictive-open.sb	3.36 kB	0 B
./bundle/sandbox-macos-restrictive-proxied.sb	3.56 kB	0 B
./bundle/sandbox-macos-strict-open.sb	4.82 kB	0 B
./bundle/sandbox-macos-strict-proxied.sb	5.02 kB	0 B

_{compressed-size-action}

[mattKorwel]

LGTM