Project: Performance powered by OTel [POTEL]

[stephanie-anderson]

Project Board

See our project board to track this initiative across all SDKs

Web Frontend SDKs

Beta Give feedback

1. Performance powered by OTel (Node.js) sentry-javascript#8534
   12 of 12
   +0
   
2. Electron SDK

Web Backend SDKs

Beta Give feedback

1. Performance powered by OTel sentry-python#2251
   Enhancement Integration: OpenTelementry (OTel) Triaged +3
   
2. [elixir] Performance powered by OTel sentry-elixir#538
   0 of 1
   Type: Enhancement +1
   
3. Go SDK
4. Ruby SDK
5. Java: POTel Research sentry-java#3232
   0 of 3
   Platform: Java +1
   
6. .NET SDK powered by OTel sentry-dotnet#3159
   OpenTelemetry Product: Tracing +2
   
7. PHP SDK
8. Rust
9. Native
10. Unity
11. Unreal

Mobile SDKs

Beta Give feedback

1. react-native
2. android
3. iOS
4. Maui
5. Dart/flutter

Description

At the time of Sentry's initial performance product development, OpenTelemetry was in the nascent stages of its lifecycle and was not yet optimized for our requirements. Nevertheless, we maintained similarities in our data models and paradigms with OpenTelemetry. Since then, OpenTelemetry has significantly matured, passed the test of time, and has been generally available (GA) for over a year. It now boasts an extensive ecosystem of integrations spanning multiple technologies, including databases, queues, and protocols.

This maturity means that now is the time for us to rework our Performance Monitoring to use OTel under the hood. This way, we can leverage all the functionality from the OTel ecosystem, and overall better align with the broader ecosystem.

Personas

1. Normal Sentry User: They don't know, or really care about the underlying tech providing performance telemetry.
   i. Only using Sentry SDK, OTel under the hood and not that interested
   ii. Top Priority this is define the vast majority of our users today, at least those who get value from our Performance products
2. Sentry & OTel User: They are using the Sentry SDK and OTel SDK in parallel. They know and care about OTel.
   i. Use OTel SDK/APIs directly for instrumentation
   ii. Span Processor 2.0
   iii. 2nd Priority
3. Sentry user with 3rd part libraries that produce OTel spans natively
   i. they are very likely unaware of OTel, at least it is plausible
   ii. not a very common use case but ideally we could consume those spans as well
   iii. lowest prio

👩‍💻 User Stories

• As a developer, I want Sentry to automatically instrument all core APIs of a platform and a wide array of third-party libraries like databases, queues, and frameworks so that my traces are comprehensive and complete.
• As a developer, I don't want to choose between two tracing implementations (Sentry, vs. OpenTelemetry).
• As a developer, I want Sentry to provide me with the best developer experience (DX) possible and provide a curated solution even if OpenTelemetry is used in the background.
• As Sentry, we don't want to degrade existing capabilities or rely on OpenTelemetry to fix bugs.
• As Sentry, we want to make this switch without requiring changes to downstream services.

🎯 Goals of This Project

• Increasing our technology currency, by leveraging OTel's instrumentations for performance
• For the initial work, we want to strive to optimize for easy setup & usage.
• Ensure a consistent user experience - it should not feel much different to users than the current SDK experience
• Do not lose functionality - in places where our instrumentation may exceed what OTel currently offers, we want to either keep some of our own instrumentation, and/or in the medium term upstream improvements to the OTel instrumentations
• Focus on automatic instrumentation
• TLDR: Users continue to use Sentry.init() and everything just works™️.

🚫 Non-Goals of This Project

• Exposing Otel internals & providing more hooks etc. for users to manually add Otel stuff may come at a later point.
• The public API of the SDK will also not change, it will remain the same for the prototype.
• We will not focus on manual instrumentation, hence the public API is also less important.

Clarifications

• The decision to use OpenTelemetry to power our framework instrumentation was already made
• While the API should be consistent on frontend and backend, we will still stick to Sentry on the frontend as OTel Mobile and Browser are still behind.
• The goal of this initiative is not to create an OTLP span ingest endpoint or to support OpenTelemetry to claim support (we support OTel already today).
• We chose this direction because OTel provides excellent tech coverage and a well-defined well-known nomenclature and API for adding manual instrumentation (Traces, Spans, Tags, etc.).
• We want to curate how we distribute OpenTelemetry to our users. This means they will still install the Sentry SDK, not vanilla OTel.
• Sentry has features Otel does not have, so our API will likely be a superset of the OTel API, and that is ok as long as we make sure that we can still stay up-to-date with the OTel project.
• When designing the API, please refer to the OpenTelemetry Tracing Spec.
• We want to minimize dependencies to other Sentry initiatives to avoid deadlocks - as such, the goal is to transform the emitted data to the Sentry schema as much as possible. There should be no changes on the backend required but we can seek out synergies when changes are due anyways.
• Eventually and one day we may natively support OTel schema and OTLP on ingest.

Notion Documents

• https://www.notion.so/sentry/OTel-SDK-Platform-7f1900c5c1b04870bdc0f2cc8dd4d929#65591679b10a4573b490d756d2c80cba
• https://www.notion.so/sentry/DACI-POTel-Node-API-Surface-b89d4da746034863b48e89a571f330a3

PR/FAQ Document

https://www.notion.so/sentry/PR-FAQ-Sentry-Tracing-Standardizes-on-OpenTelemetry-for-Tracing-3e381b48f219403c9fd7ff69f6ea4af6

Discuss

#proj-performance-powered-by-otel

OTel Instrumentations

Node.js

• https://opentelemetry.io/ecosystem/registry/?s=node&component=instrumentation&language=js
• https://github.com/open-telemetry/opentelemetry-js-contrib/tree/main/plugins/node

Python

• https://opentelemetry.io/ecosystem/registry/?s=&component=instrumentation&language=python

FAQ

Why do we want users not longer have to think about transactions/hubs/scopes?

Because they are an implementation detail of Sentry's SDK. Transactions on one had are a bit more front and center but for a user to achieve a simple measurement (Span) we shouldn't require them to understand all these concepts. We internally need to abstract away this complexity and let the user measure what they want simple as e.g.: startSpan/finishSpan and/or trace callback function.

Why OTEL - or - How we want to use OTEL?

Today the OTEL setup is unnecessarily complex and it feels like either you have to use Sentry or use OTEL - The end result has to be that there is only one way and to achieve the highest possible instrumentation we have to use OTEL underneath to cover all integrations possible.

How do we make sure that the user always gets a measurement?

At any point in time - we want to ensure whenever the user measures something - it ends up in Sentry. Specific example even if today there is no ongoing transaction we create one under the hood and send it to Sentry.