Skip to content

Conversation

@shyamnamboodiripad
Copy link
Contributor

@shyamnamboodiripad shyamnamboodiripad commented Sep 5, 2025

See TELEMETRY.md included in this PR for details.

Microsoft Reviewers: Open in CodeFlow

@github-actions github-actions bot added the area-ai-eval Microsoft.Extensions.AI.Evaluation and related label Sep 5, 2025
@shyamnamboodiripad shyamnamboodiripad force-pushed the telemetry branch 4 times, most recently from 5d2d178 to d2bb06e Compare September 8, 2025 09:01
@shyamnamboodiripad shyamnamboodiripad force-pushed the telemetry branch 3 times, most recently from 116b6c2 to 4164ad2 Compare October 6, 2025 08:22
@shyamnamboodiripad shyamnamboodiripad force-pushed the telemetry branch 4 times, most recently from 6eb98e8 to 0c6a834 Compare October 23, 2025 03:49
@shyamnamboodiripad shyamnamboodiripad force-pushed the telemetry branch 5 times, most recently from 513ea38 to 09839c2 Compare October 30, 2025 20:59
@shyamnamboodiripad shyamnamboodiripad marked this pull request as ready for review October 30, 2025 21:02
@shyamnamboodiripad shyamnamboodiripad requested review from a team as code owners October 30, 2025 21:02
Copilot AI review requested due to automatic review settings October 30, 2025 21:02
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds telemetry collection functionality to the aieval .NET tool to help understand usage patterns and improve the user experience. The telemetry is anonymous and only collected when using the command-line tool, not when using the evaluation libraries in code.

  • Adds comprehensive telemetry infrastructure with opt-out capability
  • Renames and refactors utilities for better consistency across the codebase
  • Includes detailed documentation explaining what data is collected and privacy commitments

Reviewed Changes

Copilot reviewed 40 out of 40 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/Libraries/Microsoft.Extensions.AI.Evaluation.Console/Telemetry/TelemetryHelper.cs Core telemetry collection class with Application Insights integration
src/Libraries/Microsoft.Extensions.AI.Evaluation.Console/Telemetry/TelemetryConstants.cs Constants and configuration for telemetry collection including opt-out logic
src/Libraries/Microsoft.Extensions.AI.Evaluation.Console/TELEMETRY.md Documentation explaining telemetry collection, privacy, and opt-out procedures
src/Libraries/Microsoft.Extensions.AI.Evaluation.Console/Program.cs Updated main program to initialize telemetry and display first-run messages
src/Libraries/Microsoft.Extensions.AI.Evaluation.Console/Commands/*.cs Updated command classes to integrate telemetry reporting
src/Libraries/Microsoft.Extensions.AI.Evaluation/Utilities/BuiltInMetricUtilities.cs Renamed from BuiltInEvaluatorUtilities and added new metadata constants
Various test and project files Updated references and cleaned up pragma warnings for consistency

@shyamnamboodiripad shyamnamboodiripad force-pushed the telemetry branch 2 times, most recently from c4aef14 to b3b72a1 Compare October 30, 2025 21:53
This was referenced Nov 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-ai-eval Microsoft.Extensions.AI.Evaluation and related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants