Skip to content

[AI Evaluation] Include reporting for token counts and latency #5970

@shyamnamboodiripad

Description

@shyamnamboodiripad

The evaluation library currently includes evaluators that score AI responses for various metrics like Coherence, Fluency etc. as well as for metrics defined by custom IEvaluators.

It would be great to also capture stats such as number of tokens present in the response, latency of the response and whether or not the response was fetched from cache v/s via an LLM call etc. in the report.

Additionally, it would also be a good idea to include similar cache hit, latency and token count measurements for each evaluation that is performed using an LLM.

FYI @peterwald

Metadata

Metadata

Labels

area-ai-evalMicrosoft.Extensions.AI.Evaluation and related

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions