Skip to content
This repository was archived by the owner on Aug 14, 2024. It is now read-only.

feat: Initial OpenTelemetry Spec #686

Merged
merged 14 commits into from
Oct 17, 2022
1 change: 1 addition & 0 deletions src/components/sidebar.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -226,6 +226,7 @@ export default () => {
<SidebarLink to="/sdk/performance/" title="Performance">
<SidebarLink to="/sdk/performance/span-operations/">Span Operations</SidebarLink>
<SidebarLink to="/sdk/performance/dynamic-sampling-context/">Dynamic Sampling Context</SidebarLink>
<SidebarLink to="/sdk/performance/opentelemetry/">OpenTelemetry Support</SidebarLink>
</SidebarLink>
<SidebarLink to="/sdk/event-payloads/" title="Event Payloads">
<SidebarLink to="/sdk/event-payloads/transaction/">
Expand Down
89 changes: 89 additions & 0 deletions src/docs/sdk/performance/opentelemetry/index.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
---
title: "OpenTelemetry Support"
---

<Alert level="warning">

This page is under active development. Specifications are not final and subject to change.

"Proof is in the progress" - Drake

</Alert>

This document details Sentry's work in integrating and supporting [OpenTelemetry](https://opentelemetry.io/), the open standard for metrics, traces and logs. In particular, it focuses on the integration between [Sentry's performance monitoring product](https://docs.sentry.io/product/performance/) and [OpenTelemetry's tracing spec](https://opentelemetry.io/docs/concepts/signals/traces/).

## Background

When Sentry performance monitoring was initially introduced, OpenTelemetry was in early stages. This lead to us adopt a slightly different model from OpenTelemetry, notably we have this concept of transactions that OpenTelemetry does not have. We've described this, and some more historical background, in our <Link to="/sdk/research/performance/">performance monitoring research document</Link>.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would just mention that Sentry and OpenTelemetry have conceptual differences. This is normal and true for each vendor with an SDK that predates OTel. It doesn't matter why it is like that today. It's more important to a reader what the differences are.


## Approach

TODO: Talk about the approach we are using, based on Matt's hackweek project - https://github.com/getsentry/sentry-ruby/pull/1876
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be good if @sl0thentr0py and @mjq-sentry would chime in a few words about the approach @mjq-sentry took. Can you guys do this?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this really need to go into the docs?
Should this document our way to a solution or the solution?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should document the solution, I'll work with the team to get this in there.


## Span Protocol

import "./span-protocol.mdx"

## Transaction Protocol

There is no concept of a transaction within OpenTelemetry, so we rely on promoting spans to become transactions. The span `description` becomes the transaction `name`, and the span `op` becomes the transaction `op`. Therefore, OpenTelemetry spans must be mapped to Sentry spans before they can be promoted to become a transaction.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When talking about this, I would make it clearer on what spans are Otel spans and what spans are Sentry spans.

I can kinda figure it out, but it feels like I am guessing.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 - maybe an illustration. I can help if needed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll work on clarifying the wording, let me see what I can do about the illustration.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The span description becomes the transaction name

the span description is in some cases extremely high cardinality. Are we going to introduce more transaction name sources to mitigate this?


**We should not set tags on transactions. Instead, we should populate the `attributes` field on the `otel` context, <Link to="./#opentelemetry-context">see below</Link>**

## OpenTelemetry Context

Aside from information from Spans and Transactions, OpenTelemetry has meta-level information about the SDK, resource, and service that generated spans. To track this information, we generate a new OpenTelemetry Event Context.

The existence of this context on an event (transaction or error) is how the Sentry backend will know that the incoming event is an OpenTelemetry event.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can link to event payload here and move the Otel context to event payload page.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm actually going to leave this in here so we don't confuse others reading the develop docs (new hires, other teams, etc.). once we are comfortable with this new otel context schema, let's move this outside into the main event payload page.


It uses the following schema:

```ts
type Primitive = number | string | boolean | bigint;

interface Attributes<T extends Primitive> {
[key: string]: T | Array<T> | Record<string, T>;
}

interface OpenTelemetryContext {
type?: "otel",

// https://github.com/open-telemetry/opentelemetry-proto/blob/724e427879e3d2bae2edc0218fff06e37b9eb46e/opentelemetry/proto/trace/v1/trace.proto#L174-L186
attributes?: Attributes;

// https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/resource/semantic_conventions/README.md#service
service?: {
// from: service.name
name: string;
// from: service.namespace
namespace?: string;
// from: service.instance.id
instance_id?: string;
// from: service.version
version?: string;
};

// https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/resource/semantic_conventions/README.md#telemetry-sdk
otel_sdk?: {
// from: telemetry.sdk.name
name?: string;
// from: telemetry.sdk.language
language?: SDKLanguage;
// from: telemetry.sdk.version
version?: string;
// from: telemetry.auto.version
auto_version?: string;
};

// Rest of attribute keys excluding service and sdk keys
// https://github.com/open-telemetry/opentelemetry-proto/blob/724e427879e3d2bae2edc0218fff06e37b9eb46e/opentelemetry/proto/resource/v1/resource.proto#L32
[key: string]?: string,
}
```

The reason sdk and service are split are so they can be indexed as top level fields in the future for easier usage within Sentry.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be good to capture how are we planning to fill in the service fields

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is done using by grabbing them off the OpenTelemetry resource, but I can do this!


## SDK Spec

- SpanProcessor: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/sdk.md#span-processor
- Propogator: https://opentelemetry.io/docs/reference/specification/context/api-propagators/
Loading