[Tracing] trace-flag of header 'Traceparent' is always '01' no matter the value of tracing_sampling_rate.

[mong0520]

Is there an existing issue for this?

• I have searched the existing issues

Kong version ($ kong version)

Kong 3.4.0

Current Behavior

Regarding to the definition of header Traceparent, the last field should be represented this span is recored or not.

For example, no matter the value of tracing_sampling_rate is set, kong always sets the the trace-flag of Traceparent to 01.

It causes the behavior of Kong is doing the sampling correctly regarding to tracing_sampling_rate, but always tells the upstream 'this span should be recorded'. Child span respects parent span's flag to determine if this span should be recored or not, so since kong always set it to 01, it makes child span will be always recorded.

Traceparent: 00-eb00cf0993a96552c35f60db0c6af774-43573e953680fb1a-01

Expected Behavior

The last flag should 00 if kong does not record this span.

Steps To Reproduce

1. Enable OpenTelemetry plugin
2. Set tracing_sampling_rate to 0.5
3. Setup an upstream and monitor the incoming header of Traceparent
4. Found the trace-flag field is always 01 no matter kong recored the span or not.

Anything else?

the value of balancer_span.should_sample is always true(ref: https://github.com/Kong/kong/blob/3.4.0/kong/plugins/opentelemetry/handler.lua#L129), so that the trace-flag is always 01 in this function. (ref: https://github.com/Kong/kong/blob/3.4.0/kong/tracing/propagation.lua#L589)

@samugi
I am not sure if it related to #11468

No response

[samugi]

@mong0520 Hi, thanks for reporting this. I can already say it is not related to the PR you have linked because that has not been released yet. I will look into this and provide an update.

[mong0520]

@samugi
I just did a very rough workaround on my side and it works as expectation as the trace-flag shows correctly.
The trace-flag of traceparent header will be 00 or 01 by this fix and it depends on the root span is traced or not. ( tracing_sampling_rate)

But I thought it is differently not a good fix , just FYI and try to reduce your time.
file path: kong/plugins/opentelemetry/handler.lua

[samugi]

Hi @mong0520 thanks for the additional details. I believe this should've been taken care of by #11468 - would it be possible for you to verify that the issue you encountered is not reproducible with that PR? That fix will be included in the next minor release.

[mong0520]

@samugi
This issue is fixed on my local testing with Kong 3.4.1, thank you!

[samugi]

awesome, thanks for confirming @mong0520