Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: instrument pyo3-opentelemetry #391

Merged
merged 5 commits into from
Jan 18, 2024

Conversation

erichulburd
Copy link
Contributor

@erichulburd erichulburd commented Nov 14, 2023

Goals

The goal of this MR is to support collection of Rust traces by Python end users. Specifically this entails:

  1. Connecting traces from Python to Rust.
  2. Ensuring all network API calls are instrumented with tracing (QCS API client already has support for this with the tracing-opentelemetry feature).
  3. Supporting the ability of Python users to configure and initialize Rust tracing subscribers.

Design & Implementation

  • The pyo3_opentelemetry::pypropagate macro will get the OpenTelemetry context from Python, deserialize it, and set it as the current context on the Rust side. This ensures traces initialized on the Python side are connected to traces within Rust.
  • You will see many calls to FutureExt::with_current_context on the Rust side. This ensures that any Rust futures propagate their traces across async / await. This is especially relevant for network calls.
  • In order to configure and initialize a tracing subscriber from Python, we add the tracing_subscriber module from pyo3-tracing-subscriber (see crates/python/src/lib.rs::qcs_sdk). We also autogenerate stubs for this Python module in crates/python/build.rs (stub files are formatted with cargo make format-tracing-subscriber).

@erichulburd erichulburd force-pushed the instrument_otel branch 3 times, most recently from 9185912 to fe7f686 Compare November 15, 2023 23:17
Copy link
Contributor

@MarquessV MarquessV left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this! A couple of comments, but mostly looks like issues with the type hints.

crates/python/pyproject.toml Outdated Show resolved Hide resolved
crates/python/Cargo.toml Outdated Show resolved Hide resolved
crates/python/qcs_sdk/tracing_subscriber/__init__.pyi Outdated Show resolved Hide resolved
crates/python/src/compiler/quilc.rs Outdated Show resolved Hide resolved
crates/python/src/executable.rs Show resolved Hide resolved
crates/python/src/lib.rs Outdated Show resolved Hide resolved
crates/python/src/py_sync.rs Show resolved Hide resolved
@MarquessV
Copy link
Contributor

Oh - when testing I also noticed that poetry.lock needs to be updated.

@MarquessV
Copy link
Contributor

@erichulburd The checks are passing on this PR now. Would you like me to rebase, merge, and release an RC?

@MarquessV MarquessV merged commit 1cf88e9 into rigetti:main Jan 18, 2024
1 of 2 checks passed
@erichulburd erichulburd deleted the instrument_otel branch January 19, 2024 18:48
MarquessV added a commit that referenced this pull request Jan 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants