Multiple duplicated records in Jaeger

[asyncee]

Hello.

I have created a simple demo project in Python that uses fastapi, requests and corresponding instrumentation libraries. The project consists of two simple apps. First application sends request to second application to get some json payload back and then displays it on the screen to the user. All communications are instrumented via instrumentation libraries:

• opentelemetry-instrumentation-requests
• opentelemetry-instrumentation-fastapi

I have two exporters: Jaeger one and console exporter (for debugging).

When i open Jaeger interface, i see multiple duplicate entries for the same span and a warning message "Duplicate span ID", but json that is written to sdtout is valid and does not contain any duplicate ids. Jaeger is deployed in Docker container and using last version.

I have created demo project to help you reproduce the problem: https://github.com/asyncee/opentelemetry-issue/tree/master

Following screenshot demonstrates that every span is duplicated (app_a / and app_b /welcome):

Following screenshot demonstrates warnings:

Trace created by console exporter:

{
    "name": "/welcome",
    "context": {
        "trace_id": "0x2f42e1c358ec8f0901fb3df0715b52f5",
        "span_id": "0xce26fccf215dfd48",
        "trace_state": "{}"
    },
    "kind": "SpanKind.CLIENT",
    "parent_id": "0x1141b51b2ec62f64",
    "start_time": "2020-08-26T15:02:36.347670Z",
    "end_time": "2020-08-26T15:02:36.359535Z",
    "status": {
        "canonical_code": "OK"
    },
    "attributes": {
        "component": "http",
        "http.method": "GET",
        "http.url": "http://localhost:8001/welcome",
        "http.status_code": 200,
        "http.status_text": "OK"
    },
    "events": [],
    "links": [],
    "resource": {}
}
{
    "name": "get_welcome_message",
    "context": {
        "trace_id": "0x2f42e1c358ec8f0901fb3df0715b52f5",
        "span_id": "0x1141b51b2ec62f64",
        "trace_state": "{}"
    },
    "kind": "SpanKind.INTERNAL",
    "parent_id": "0x93f7571142329617",
    "start_time": "2020-08-26T15:02:36.347574Z",
    "end_time": "2020-08-26T15:02:36.359591Z",
    "status": {
        "canonical_code": "OK"
    },
    "attributes": {},
    "events": [],
    "links": [],
    "resource": {}
}
{
    "name": "/ asgi.http.send",
    "context": {
        "trace_id": "0x2f42e1c358ec8f0901fb3df0715b52f5",
        "span_id": "0x11bd45da87f24542",
        "trace_state": "{}"
    },
    "kind": "SpanKind.INTERNAL",
    "parent_id": "0x93f7571142329617",
    "start_time": "2020-08-26T15:02:36.360270Z",
    "end_time": "2020-08-26T15:02:36.360558Z",
    "status": {
        "canonical_code": "OK"
    },
    "attributes": {
        "http.status_code": 200,
        "type": "http.response.start"
    },
    "events": [],
    "links": [],
    "resource": {}
}
{
    "name": "/ asgi.http.send",
    "context": {
        "trace_id": "0x2f42e1c358ec8f0901fb3df0715b52f5",
        "span_id": "0xd1ff9dbd6eb6b48e",
        "trace_state": "{}"
    },
    "kind": "SpanKind.INTERNAL",
    "parent_id": "0x93f7571142329617",
    "start_time": "2020-08-26T15:02:36.360623Z",
    "end_time": "2020-08-26T15:02:36.360686Z",
    "status": {
        "canonical_code": "OK"
    },
    "attributes": {
        "type": "http.response.body"
    },
    "events": [],
    "links": [],
    "resource": {}
}
{
    "name": "/ asgi",
    "context": {
        "trace_id": "0x2f42e1c358ec8f0901fb3df0715b52f5",
        "span_id": "0x93f7571142329617",
        "trace_state": "{}"
    },
    "kind": "SpanKind.SERVER",
    "parent_id": null,
    "start_time": "2020-08-26T15:02:36.346879Z",
    "end_time": "2020-08-26T15:02:36.360705Z",
    "status": {
        "canonical_code": "OK"
    },
    "attributes": {
        "component": "http",
        "http.scheme": "http",
        "http.host": "127.0.0.1:8000",
        "host.port": 8000,
        "http.flavor": "1.1",
        "http.target": "/",
        "http.url": "http://127.0.0.1:8000/",
        "http.method": "GET",
        "http.server_name": "127.0.0.1:8000",
        "http.user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.108 Safari/537.36",
        "net.peer.ip": "127.0.0.1",
        "net.peer.port": 51389,
        "http.route": "/"
    },
    "events": [],
    "links": [],
    "resource": {}
}

Thanks!

[asyncee]

Zipkin backend does not have this problem, so i will switch to that for my research needs.

[codeboten]

Thanks for following up @asyncee, just to close the loop, I think this issue was a duplicate of #981 which was just recently fixed in the jaeger exporter.