Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Manually set the parentSpanId #89

Open
Fi3 opened this issue Jan 19, 2024 · 6 comments
Open

Manually set the parentSpanId #89

Fi3 opened this issue Jan 19, 2024 · 6 comments

Comments

@Fi3
Copy link

Fi3 commented Jan 19, 2024

Feature Request

Looking for a way to manually set the parentSpanId on span creation (someting like otel.name but for parentSpanId)

Motivation

I have several service that communicate via a custom binary protocol and I do not know how to pass context. A lot of messages have a request_id using it + the originator of the request + a rounded timestamp I should be able to have a consistent parentSpanId trough requests.

Very new to otel so maybe I'm missing something. What I'm looking for is the ability to have distributed traces with Jaeger witout using http.

@Fi3
Copy link
Author

Fi3 commented Jan 22, 2024

I also look for a way to set the span id. The idea is to deterministically set a span id (using my request id) so that in the service that I call (that know the request id) I know the parent span id and I can set is with set_parent. There is a way to do that? I can not find it.

@Fi3
Copy link
Author

Fi3 commented Jan 22, 2024

Looking at the code the most immediate way is to add in layer.rs a layer implementor that instead of using self.tracer.new_span_id() to set the span_id on on_new_span like OpenTelemetryLayer it use a field on the span.

Maybe instead of having a brand new implemetor we can just use OpenTelemetryLayer and look for a specific field. If we find it we use it as span id otherwise we fallback to the already existent method.

Does this approach make any sense?

In case I can open a PR

@jtescher
Copy link
Collaborator

You can't set span ids yourself in opentelemetry without implementing a custom id generator, however you can set parent span, which will associate the new span with the same trace which is generally advised. This would be an example of extracting a parent from headers or other kv data to associate the parent https://github.com/tokio-rs/tracing-opentelemetry/blob/v0.1.x/examples/opentelemetry-remote-context.rs#L32-L34

@Fi3
Copy link
Author

Fi3 commented Jan 23, 2024

From what I understood jaeger use parent child relations in order to render traces. Not having the possibility to pass additional data via the binary protocol that I'm using, I want to set span ids deterministically using request ids and timestamp. From what I understood this will require patching tracing-opentelemetry. I have 3 questions:

  1. To do that I need to patch tracing-opentelemetry or not?
  2. This approach have some risk or downside that I can not see?
  3. if I go with this approach are you interested in a PR that allow setting custom span ids?

I can not just use the span id as request id in a lot of cases.

@mladedav
Copy link
Contributor

I am not exactly sure, but I think you could try and create your own Tracer and call with_tracer on the OpenTelemetryLayer. It would let you create your own span IDs.

I think there are risks concerning the the uniqueness of span ID, e.g. if a client retries a request with the same request ID, you might end up with two spans with the same ID which could cause issues in your tracing backend.

Personally, I would advise you against this though, you should be able to pass the context to the child so that you can set the parent to the apriori generated span context.

@Fi3
Copy link
Author

Fi3 commented Feb 26, 2024

at the end I vendored tracing-opentelemetry an add a very little patch. I can have collisions and also request that should be grouped but end up not grouped. They are statistically very unlikely and this is acceptable, it do not mess with back-end and I do not need to have 100% accuracy.

I patched layer.rs adding:

struct IdVisitor {
    id: Option<otel::SpanId>,
}

And modifying:

impl<S, T> Layer<S> for OpenTelemetryLayer<S, T>
where
    S: Subscriber + for<'span> LookupSpan<'span>,
    T: otel::Tracer + PreSampledTracer + 'static,
{
        fn on_new_span(&self, attrs: &Attributes<'_>, id: &span::Id, ctx: Context<'_, S>) {
            ...
            let mut id_getter = IdVisitor { id: None };
            attrs.values().record(&mut id_getter);
            let span_id = match id_getter.id {
                Some(span_id) => span_id,
                None => self.tracer.new_span_id(),
            };
            ....
            let mut builder = self
                .tracer
                .span_builder(attrs.metadata().name())
                .with_start_time(crate::time::now())
                // Eagerly assign span id so children have stable parent id
                .with_span_id(span_id);
            ....   
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants