[Executor] Update openai tokens to trace

[lumoslnt]

Description

This pull request primarily introduces changes to the src/promptflow/promptflow/_core/tracer.py file. The main focus of the changes is to enhance the tracing mechanism by adding a TokenCollector class for collecting OpenAI tokens associated with spans, and integrating it into the existing tracing flow. The changes also include a minor import modification for the OpenTelemetry library.

Main Changes:

• src/promptflow/promptflow/_core/tracer.py: Introduced a new class TokenCollector to collect OpenAI tokens associated with spans. The class provides methods to collect tokens, merge tokens for parent spans, and retrieve tokens for a given span id.
• src/promptflow/promptflow/_core/tracer.py: The enrich_span_with_output function now also sets token attributes to the span if any tokens are associated with the span id.
• src/promptflow/promptflow/_core/tracer.py: The wrapped function (both synchronous and asynchronous versions) now collects OpenAI tokens if the trace type is LLM, and collects tokens for the parent span before returning the output.

All Promptflow Contribution checklist:

• The pull request does not introduce [breaking changes].
• CHANGELOG is updated for new features, bug fixes or other significant changes.
• I have read the contribution guidelines.
• Create an issue and link to the pull request to get dedicated review from promptflow team. Learn more: suggested workflow.

General Guidelines and Best Practices

• Title of the pull request is clear and informative.
• There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

• Pull request includes test coverage for the included changes.

[github-actions]

SDK CLI Global Config Test Result devs/linatang/update_openai_tokens_to_trace

2 tests   2 ✅  49s ⏱️
1 suites  0 💤
1 files    0 ❌

Results for commit b6f9c5d.

♻️ This comment has been updated with latest results.

[github-actions]

Executor Unit Test Result devs/linatang/update_openai_tokens_to_trace

705 tests   705 ✅  44s ⏱️
  1 suites    0 💤
  1 files      0 ❌

Results for commit b6f9c5d.

♻️ This comment has been updated with latest results.

[github-actions]

promptflow SDK CLI Azure E2E Test Result devs/linatang/update_openai_tokens_to_trace

  4 files    4 suites   3m 21s ⏱️
129 tests 110 ✅ 19 💤 0 ❌
516 runs  440 ✅ 76 💤 0 ❌

Results for commit b6f9c5d.

♻️ This comment has been updated with latest results.

[github-actions]

Executor E2E Test Result devs/linatang/update_openai_tokens_to_trace

187 tests   185 ✅  3m 48s ⏱️
  1 suites    2 💤
  1 files      0 ❌

Results for commit b6f9c5d.

♻️ This comment has been updated with latest results.

[github-actions]

SDK CLI Test Result devs/linatang/update_openai_tokens_to_trace

   12 files     12 suites   39m 3s ⏱️
  419 tests   411 ✅  8 💤 0 ❌
1 676 runs  1 644 ✅ 32 💤 0 ❌

Results for commit b6f9c5d.

♻️ This comment has been updated with latest results.