Project: Send trace origin

[philipphofmann]

Description

Trace origin indicates what created a trace or a span. Not all traces and spans contain enough information to tell whether the user or what precisely in the SDK created it. Origin solves this problem. Origin can be sent with the trace context or spans. The SDKs must set this value automatically. It is not expected to be set manually by users. For more information, visit the develop docs. If the spec needs further clarification, you can ping @philipphofmann.

As origin for trace context and spans is optional, every SDK can decide whether to start sending origin for everything or just selected transactions and spans. This issue is important for internal data analysis to understand the quality of our traces; see internal Notion doc.

Project Overview

• Project board
• Task list

Web Frontend SDKs

Beta Give feedback

1. JavaScript
2. Send span origin sentry-electron#696
   Status: Untriaged +1
   

Web Backend SDKs

Beta Give feedback

1. Send span origin sentry-java#2692
   Platform: Java +1
   
2. Send span origin sentry-python#2244
   Triaged +1
   
3. Send span origin sentry-go#668
   Status: Backlog Type: Improvement +2
   
4. Send span origin sentry-php#1566
   Type: Improvement +1
   
5. Send span origin sentry-laravel#729
   Type: Improvement +1
   
6. Send span origin sentry-symfony#745
   Type: Improvement +1
   
7. Send span origin sentry-ruby#2068
   +0
   
8. Send trace origin sentry-rust#681
   +0

Mobile SDKs

Beta Give feedback

1. Cocoa
2. Send span origin sentry-java#2692
   Platform: Java +1
   
3. Send span origin sentry-dart#1420
   Platform: Dart performance +2
   
4. Send span origin sentry-react-native#3052
   1 of 2
   Platform: React-Native +1
5. Send span origin sentry-unity#1340
   Feature +1
6. Send trace origin sentry-kotlin-multiplatform#228
   Platform: KMP +1

GDX SDKs

Beta Give feedback

1. Send span origin sentry-unity#1340
   Feature +1
2. Unreal doesn't have performance API yet
3. Send span origin sentry-dotnet#2472
   Feature +1
   

[HazAT]

Is there something like this in OTEL?

[philipphofmann]

@HazAT, I don't think so. When I wrote the RFC getsentry/rfcs#73, I checked the OTEL spec and didn't find anything similar, but I could be wrong as I'm not that familiar with the spec. @AbhiPrasad, as you quote OTEL regularly, do you know if something like the trace orign exists? I guess not cause you would have brought it up in getsentry/rfcs#73.

There is a somewhat similar HTTP header origin I used as a bit of inspiration for trace origin.

[jamescrosswell]

@philipphofmann Sentry's OpenTelemetry integrations take OpenTelemetry spans and create Sentry transactions/spans for these in a SpanProcessor.

It's hard to tell where those spans came from in these cases though. Were they created automatically by OpenTelemetry or do those spans correspond to spans that users of the OpenTelemetry SDK have instrumented manually? Since we're one step removed from the instrumentation, I'm not sure how we determine this.

Has this been solved in any of the existing SDK implementations of Trace Origin already?

[mydea]

FYI this is now released in v7.66.0 of the JS SDK!

[philipphofmann]

@jamescrosswell, sorry for the late reply. I was on parental leave for two months. As far as I know, no existing SDK has solved the problem with OpenTelemetry integrations. Could you let me know if you've already solved this problem? If not, I can have a look and add some context to the develop docs.

[jamescrosswell]

Hi @philipphofmann, no problem.

No I haven't/can't see a solution to this. About the best I can think of is that in addition to Manual and Automatic, we add Unknown as a possible value... and in the case of OpenTelemetry instrumented spans, these would always have TraceOrigin.Unknown.

[philipphofmann]

@jamescrosswell, the OpenTelemetry instrumented spans should have a span operation, and when creating these spans, we know they stem from OpenTelemetry. Ideally, we should be able to do something like auto.db.otel, auto.http.otel, etc. Unknown doesn't add any value.

[jamescrosswell]

Hm, OK thanks @philipphofmann. In the .NET implementation of OpenTelemetry, spans basically start out as an Activitiy and Activities do have an OperationName... however this can be set even for spans that are instrumented manually.

We could check to see if the OperationName corresponded to a value that would be used by automatic instrumentation, as long as we could get a list of all the values that might be set for this via automatic instrumentation. I'm not sure if such a list exists anywhere.

The semantic conventions documentation talks about keys for all sorts of attributes. I think lots of those are used in the Activity.Tags though... it's hard to tease out which ones could/would be used as the value for the Span.OperationName. We do have a list of those (kind of) as a bunch of C# constants... but it doesn't look like those values are what we're after.

Did you have a specific set of Operation Names in mind that you'd seen somewhere?

[philipphofmann]

Trace origin uses the category of span operations. When you create spans from OTel, you must extract span operations, which also need to have a span category. Looking at the bunch of C# constants I would say you could pick the first part of these constants. For example, db.mssql.instance_name would have the span category db. Then you simply append the name of the integration that creates these spans, which in your case could be otel. So you would end up with auto.db.otel. Would that work for you, @jamescrosswell?

[jamescrosswell]

We can definitely build that, yes. I'll trust your guidance in terms of whether it's doing what we want or not 👍🏻

[philipphofmann]

We can definitely build that, yes

That's great.

I'll trust your guidance in terms of whether it's doing what we want or not

I think the best is you tag me in your PR and I will give you a green light if it LGTM, @jamescrosswell.

[smeubank]

@philipphofmann i am moving this back to planning

Can we update this epic with which SDKs have done this already? Is there a way we can check which sdks are sneing it correctly? Via looker for example

Then track the remaining SDKs so we can try to close this project

[philipphofmann]

@smeubank, thanks, for following up on this. You can check this Looker board.

I updated the task lists above.