Skip to content

Python: ApplicationInsights agent span has incorrect token numbers #1320

@jomalsan

Description

@jomalsan

I am using:

  • agent-framework v 1.0.0b251007 (But the same was true for v 1.0.0b251001)
  • AzureOpenAIResponsesClient
  • setup_observability(enable_sensitive_data=True, applicationinsights_connection_string="omitted")

What is happening:
In the latest version, each independent call to the model is being tracked as expected, with their token usage correctly attached. The agent's span then also records token usage, but is set to the values of the final llm call. This is misleading

Expected behavior is either:

  1. The agent span does not report token usage because they are reported on the model call
  2. The agent span reports the sum of tokens used in it's children spans

Screenshot showing application insights with the agent's span. The token usage matches the second chat o4-mini span's token usage:

Image

Metadata

Metadata

Labels

agentsIssues related to single agentsobservabilityIssues related to observability or telemetrypythonv1.0Features being tracked for the version 1.0 GA

Type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions