-
Notifications
You must be signed in to change notification settings - Fork 844
Add usage telemetry for aieval dotnet tool #6773
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
src/Libraries/Microsoft.Extensions.AI.Evaluation.Console/Commands/ReportCommand.cs
Outdated
Show resolved
Hide resolved
5d2d178 to
d2bb06e
Compare
d2bb06e to
67eaa61
Compare
116b6c2 to
4164ad2
Compare
6eb98e8 to
0c6a834
Compare
513ea38 to
09839c2
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds telemetry collection functionality to the aieval .NET tool to help understand usage patterns and improve the user experience. The telemetry is anonymous and only collected when using the command-line tool, not when using the evaluation libraries in code.
- Adds comprehensive telemetry infrastructure with opt-out capability
- Renames and refactors utilities for better consistency across the codebase
- Includes detailed documentation explaining what data is collected and privacy commitments
Reviewed Changes
Copilot reviewed 40 out of 40 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
src/Libraries/Microsoft.Extensions.AI.Evaluation.Console/Telemetry/TelemetryHelper.cs |
Core telemetry collection class with Application Insights integration |
src/Libraries/Microsoft.Extensions.AI.Evaluation.Console/Telemetry/TelemetryConstants.cs |
Constants and configuration for telemetry collection including opt-out logic |
src/Libraries/Microsoft.Extensions.AI.Evaluation.Console/TELEMETRY.md |
Documentation explaining telemetry collection, privacy, and opt-out procedures |
src/Libraries/Microsoft.Extensions.AI.Evaluation.Console/Program.cs |
Updated main program to initialize telemetry and display first-run messages |
src/Libraries/Microsoft.Extensions.AI.Evaluation.Console/Commands/*.cs |
Updated command classes to integrate telemetry reporting |
src/Libraries/Microsoft.Extensions.AI.Evaluation/Utilities/BuiltInMetricUtilities.cs |
Renamed from BuiltInEvaluatorUtilities and added new metadata constants |
| Various test and project files | Updated references and cleaned up pragma warnings for consistency |
src/Libraries/Microsoft.Extensions.AI.Evaluation.Console/Telemetry/TelemetryConstants.cs
Outdated
Show resolved
Hide resolved
src/Libraries/Microsoft.Extensions.AI.Evaluation.Console/Commands/ReportCommand.cs
Outdated
Show resolved
Hide resolved
c4aef14 to
b3b72a1
Compare
See TELEMETRY.md included in this PR for details.
Microsoft Reviewers: Open in CodeFlow