Skip to content

Conversation

Copy link

Copilot AI commented Jan 15, 2026

Summary

Implements 7 histogram metrics for LLM operations in the aliyun_trace component, exported via OTLP to Aliyun endpoint. All metrics include app_name tag.

Metrics Added:

  • gen_ai.client.time_to_first_token - First token latency
  • gen_ai.client.time_per_output_token - Per-token generation time (auto-calculated from duration/completion_tokens)
  • gen_ai.client.time_between_token - Inter-token timing
  • gen_ai.client.operation - Operation count
  • gen_ai.usage.usage.prompt_tokens_details.cached_tokens - Cached token usage
  • gen_ai.client.operation.duration - Total operation latency
  • gen_ai.client.token.usage - Total token consumption

Implementation:

  • MetricsClient (traceclient.py): OpenTelemetry SDK wrapper with histogram instruments, periodic export (5s), and graceful shutdown
  • Integration (aliyun_trace.py): Metrics recorded automatically in build_workflow_llm_span() and message_trace() when LLM spans are created
  • Tests (test_metrics_client.py): Unit tests covering initialization, recording, auto-calculation, and app_name tag validation

Metrics are recorded transparently when LLM operations execute in workflows or message-based apps. No configuration changes required beyond existing aliyun_trace setup.

# Metrics recorded automatically from usage data
usage_data = {
    "prompt_tokens": 100,
    "completion_tokens": 50, 
    "total_tokens": 150,
    "latency": 2.0
}
# → All 7 metrics exported with app_name tag

Screenshots

N/A - Backend metrics implementation

Checklist

  • This change requires a documentation update, included: Dify Document
  • I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
  • I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
  • I've updated the documentation accordingly.
  • I ran dev/reformat(backend) and cd web && npx lint-staged(frontend) to appease the lint gods
Original prompt

在aliyun_trace组件中增加gen_ai.client.time_to_first_token、gen_ai.client.time_per_output_token、gen_ai.client.time_between_token、gen_ai.client.operation、gen_ai.usage.usage.prompt_tokens_details.cached_tokens、gen_ai.client.operation.duration、gen_ai.client.token.usage这些metric指标埋点,指标类型都采用histogram类型,这些指标埋点是LLM类型的埋点,应该在aliyun_trace中处理,所有指标中都要有app_name这个tag


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 4 commits January 15, 2026 09:30
Co-authored-by: c3013 <246969133+c3013@users.noreply.github.com>
Co-authored-by: c3013 <246969133+c3013@users.noreply.github.com>
Co-authored-by: c3013 <246969133+c3013@users.noreply.github.com>
Co-authored-by: c3013 <246969133+c3013@users.noreply.github.com>
Copilot AI changed the title [WIP] Add histogram metrics for gen_ai client and usage in aliyun_trace Add OpenTelemetry histogram metrics for LLM operations in aliyun_trace Jan 15, 2026
Copilot AI requested a review from c3013 January 15, 2026 09:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants