Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remote resources not visible on X-Ray map #15

Closed
bonclay7 opened this issue Jan 26, 2021 · 4 comments
Closed

Remote resources not visible on X-Ray map #15

bonclay7 opened this issue Jan 26, 2021 · 4 comments

Comments

@bonclay7
Copy link
Member

Hi !

I have an application connecting to a SQL database, and since there's not yet support for database/sql auto-instrumentation, I did a custom tracing implementation.

tracer := otel.GetTracerProvider().Tracer("petlistadoptions")
_, span := tracer.Start(ctx, "mssql query")

span.SetAttributes(
  label.String("sql", sql),
  label.String("url", r.safeConnStr),
)

rows, err := r.db.Query(sql)
if err != nil {
  logger.Log("error", err)
  return nil, err
}
span.End()

The link to the database do not appear as a remote resource and doesn't show up in the trace map

image

Environment details :

  • Go 1.15
  • Running on AWS Fargate, with amazon/aws-otel-collector:latest as sidecar
  • go.opentelemetry.io/otel v0.15.0

Raw trace data

{
    "Id": "1-5ffc45c2-4b5e80c8040c405153b5c4ec",
    "Duration": 0.034,
    "LimitExceeded": false,
    "Segments": [
        {
            "Id": "7af254d953cd2c47",
            "Document": {
                "id": "7af254d953cd2c47",
                "name": "PetSearch",
                "start_time": 1610368450.014082,
                "trace_id": "1-5ffc45c2-4b5e80c8040c405153b5c4ec",
                "end_time": 1610368450.025903,
                "parent_id": "2a898f4841386dfe",
                "http": {
                    "request": {
                        "url": "http://servi-searc-8256xjtfxppf-1893318472.eu-west-1.elb.amazonaws.com/api/search?petid=005",
                        "method": "GET",
                        "user_agent": "Go-http-client/1.1",
                        "client_ip": "54.78.128.12",
                        "x_forwarded_for": true
                    },
                    "response": {
                        "status": 200
                    }
                },
                "aws": {
                    "xray": {
                        "sdk_version": "2.6.2",
                        "sdk": "X-Ray for .NET Core"
                    }
                },
                "metadata": {
                    "default": {
                        "Pets": "[{\"pettype\":\"puppy\",\"petid\":\"005\",\"price\":\"59\",\"petcolor\":\"brown\",\"cuteness_rate\":\"3\",\"availability\":\"no\",\"image\":null,\"peturl\":\"https://services-s3bucketpetadoptioncb20dce5-u0k3dtb800rk.s3.eu-west-1.amazonaws.com/puppies/p3.jpg?X-Amz-Expires=300\\u0026x-amz-security-token=IQoJb3JpZ2luX2VjENz%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCWV1LXdlc3QtMSJGMEQCIA0s%2FJhni7lUj8SmNHE%2BwqlgZx9s%2F1f2F6amHUEZMajnAiAjiyECmqKZ%2B94PPaKVh8kyZrB9jh3M1W3IrCrR%2BvLG8iroAwil%2F%2F%2F%2F%2F%2F%2F%2F%2F%2F8BEAEaDDMzOTc0MzEwMzcxNyIMUN8WYjKrVgBInf6PKrwDj%2BcHeNIr1tJ9WQkMqYrl5ytkFx3x%2FXDpz718AQLFLcnqj79gHuO%2BhQrBlLp%2BZooIGDCJzoNIzAQ%2FfZVBQ5Wdi3%2F0yYBtmme7rQ3Z6nXiihjYV2W8OPWGeTR1ANx6E4zM%2BQ%2BBPUMyWkzj542aazM9CE5TWqDmIaNw%2BkvRi7X6MoguGiQ8olXzovQCi5ugDZLxwsyi5rOEkqJahnTmQovTLETJHzAndRYGl6eapW9xRKmFCBpbrOt2tVnlpdovdCHhAvSSKAAbd1i3MHvnkCDksN0cUGYtnNo%2FKN%2FNQn0GNsZlAlyftPNQPBJXoOrCN7mtXrRksjya2GWLN9tMk2%2BeYUvEfaz7%2FhkCOnO6uXgAsmb4cP0ldOFskPyLAJ%2FyHXbpbOH35vOde31uXygvx1eZVRqIihNv6JKGMcWSGg82pqqtRpmEDddgyxNwdwTgUczn26FGWURFR%2Bo6rWCK1SN2K65Bc3oR9FJ1OpMDMGhA35phBDdbOBtI38qPGkgTjlbhywAgNvKn9aMJK6Ypy1cr%2FkH2eipzi%2FcDW6leCNsqbv8d5JCMnRbkXE7%2FuNWZlTbx1P2I2b2wQdITRvnYMNT48P8FOu0BUH8hri3555CU8KXxKl9%2BCFUKvEod0Sy4CJx6pbIpo2e2gjdLVCvG5w2LzJOsPlBK%2Fmz77tdDiRvEeXBQpPGrl0gyBs1PSusjbu73vG2xW2lpEGsaeCXLFxl1YsenyAFAvoXsg69pstGBUDFYcYfMIE8q4dNt1Rxsyjl3p3mtOR4eM5js%2FtzvYy%2F56EsEJ06cf3bggfOY03Kr3cyB08A%2BHHPPGkX72QWEkEjOirYeU51M7UasRcn%2FlMgrZgYwCg8oY1%2FX5zr9Cx6NvDQt5tnCFkCrrg7X0ejXcJFe6iQXPakT6MKbQQ%2BpKP4dDU5k\\u0026X-Amz-Algorithm=AWS4-HMAC-SHA256\\u0026X-Amz-Credential=ASIAU6GSELLS6NTJ45P5/20210111/eu-west-1/s3/aws4_request\\u0026X-Amz-Date=20210111T123410Z\\u0026X-Amz-SignedHeaders=host;x-amz-security-token\\u0026X-Amz-Signature=aea598ef6e8b4f26e6cd76e26934196c2805a84d19ac552bc03e666ce068f34c\"}]"
                    }
                },
                "service": {
                    "runtime": ".NET Core Framework",
                    "runtime_version": "3.1.10"
                },
                "subsegments": [
                    {
                        "id": "f82385679a9ae1f3",
                        "name": "Scanning DynamoDB Table",
                        "start_time": 1610368450.017318,
                        "end_time": 1610368450.022804,
                        "annotations": {
                            "Query": "petcolor:-pettype:-petid:005"
                        },
                        "subsegments": [
                            {
                                "id": "be7d4a3b16ec486a",
                                "name": "DynamoDB",
                                "start_time": 1610368450.017891,
                                "end_time": 1610368450.02277,
                                "http": {
                                    "response": {
                                        "status": 200,
                                        "content_length": 200
                                    }
                                },
                                "aws": {
                                    "select": null,
                                    "count": 1,
                                    "table_name": "Services-ddbpetadoption7B7CFEC9-P94QSTUX451H",
                                    "resource_names": [
                                        "Services-ddbpetadoption7B7CFEC9-P94QSTUX451H"
                                    ],
                                    "consistent_read": false,
                                    "consumed_capacity": null,
                                    "scanned_count": 26,
                                    "segment": 0,
                                    "limit": 0,
                                    "attributes_to_get": [],
                                    "projection_expression": null,
                                    "region": "eu-west-1",
                                    "operation": "Scan",
                                    "request_id": "Q5BOGNJR23DJ6UQSC71MV3RGV7VV4KQNSO5AEMVJF66Q9ASUAAJG",
                                    "total_segments": 0,
                                    "index_name": null
                                },
                                "namespace": "aws"
                            }
                        ]
                    }
                ]
            }
        },
        {
            "Id": "507aa886d62e8df8",
            "Document": {
                "id": "507aa886d62e8df8",
                "name": "servi-lista-pwhg5v3y9jol-1655753230.eu-west-1.elb.amazonaws.com",
                "start_time": 1610368450.0068173,
                "trace_id": "1-5ffc45c2-4b5e80c8040c405153b5c4ec",
                "end_time": 1610368450.0271156,
                "parent_id": "1adf1081d3fbd287",
                "fault": false,
                "error": false,
                "http": {
                    "request": {
                        "url": "http://servi-lista-pwhg5v3y9jol-1655753230.eu-west-1.elb.amazonaws.com/api/adoptionlist/",
                        "method": "GET",
                        "client_ip": "54.78.128.12",
                        "x_forwarded_for": true
                    },
                    "response": {
                        "status": 200,
                        "content_length": 0
                    }
                },
                "aws": {
                    "xray": {
                        "auto_instrumentation": false
                    }
                },
                "metadata": {
                    "default": {
                        "net.transport": "IP.TCP",
                        "resultCount": "",
                        "http.flavor": "1.1",
                        "http.route": "/api/adoptionlist/",
                        "timeTakenSeconds": ""
                    }
                },
                "subsegments": [
                    {
                        "id": "79e5dddc6e67ba9e",
                        "name": "mssql query",
                        "start_time": 1610368450.0068529,
                        "end_time": 1610368450.0116804,
                        "fault": false,
                        "error": false,
                        "aws": {
                            "xray": {
                                "auto_instrumentation": false
                            }
                        },
                        "metadata": {
                            "default": {
                                "url": "sqlserver://xxx:@si5zoymobm0wws.xxx.eu-west-1.rds.amazonaws.com:1433?database=adoptions",
                                "sql": "SELECT TOP 25 PetId, Transaction_Id, Adoption_Date FROM dbo.transactions"
                            }
                        }
                    }
                ]
            }
        },
        {
            "Id": "a91bb14030a5daed",
            "Document": {
                "id": "a91bb14030a5daed",
                "name": "PetSite",
                "start_time": 1610368450.001682,
                "trace_id": "1-5ffc45c2-4b5e80c8040c405153b5c4ec",
                "end_time": 1610368450.035526,
                "http": {
                    "request": {
                        "url": "http://servi-petsi-ga3fszv5gykz-1180376689.eu-west-1.elb.amazonaws.com/PetListAdoptions",
                        "method": "GET",
                        "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36",
                        "client_ip": "90.79.92.140",
                        "x_forwarded_for": true
                    },
                    "response": {
                        "status": 200
                    }
                },
                "aws": {
                    "ecs": {
                        "container": "ip-11-0-248-241.eu-west-1.compute.internal"
                    },
                    "xray": {
                        "sampling_rule_name": "Default",
                        "sdk_version": "2.9.0",
                        "sdk": "X-Ray for .NET Core"
                    }
                },
                "service": {
                    "runtime": ".NET Core Framework",
                    "runtime_version": "3.1.10"
                },
                "origin": "AWS::ECS::Container",
                "subsegments": [
                    {
                        "id": "d732014dd9edb9e7",
                        "name": "Calling PetListAdoptions",
                        "start_time": 1610368450.001949,
                        "end_time": 1610368450.029522,
                        "subsegments": [
                            {
                                "id": "1adf1081d3fbd287",
                                "name": "servi-lista-pwhg5v3y9jol-1655753230.eu-west-1.elb.amazonaws.com",
                                "start_time": 1610368450.002071,
                                "end_time": 1610368450.028931,
                                "http": {
                                    "request": {
                                        "url": "http://servi-lista-pwhg5v3y9jol-1655753230.eu-west-1.elb.amazonaws.com/api/adoptionlist/",
                                        "method": "GET"
                                    },
                                    "response": {
                                        "status": 200,
                                        "content_length": 1873
                                    }
                                },
                                "namespace": "remote"
                            }
                        ]
                    }
                ]
            }
        },
        {
            "Id": "02875c972c787511",
            "Document": {
                "id": "02875c972c787511",
                "name": "DynamoDB",
                "start_time": 1610368450.017891,
                "trace_id": "1-5ffc45c2-4b5e80c8040c405153b5c4ec",
                "end_time": 1610368450.02277,
                "parent_id": "be7d4a3b16ec486a",
                "inferred": true,
                "http": {
                    "response": {
                        "status": 200,
                        "content_length": 200
                    }
                },
                "aws": {
                    "select": null,
                    "count": 1,
                    "table_name": "Services-ddbpetadoption7B7CFEC9-P94QSTUX451H",
                    "resource_names": [
                        "Services-ddbpetadoption7B7CFEC9-P94QSTUX451H"
                    ],
                    "consistent_read": false,
                    "consumed_capacity": null,
                    "scanned_count": 26,
                    "segment": 0,
                    "limit": 0,
                    "attributes_to_get": [],
                    "projection_expression": null,
                    "region": "eu-west-1",
                    "operation": "Scan",
                    "request_id": "Q5BOGNJR23DJ6UQSC71MV3RGV7VV4KQNSO5AEMVJF66Q9ASUAAJG",
                    "total_segments": 0,
                    "index_name": null
                },
                "origin": "AWS::DynamoDB::Table"
            }
        }
    ]
}
@bonclay7 bonclay7 changed the title Remote resources not visible on X-Ray graph Remote resources not visible on X-Ray map Jan 26, 2021
@anuraaga
Copy link
Contributor

Hi @bonclay7 - what happens if you start the span with CLIENT type using trace.WithSpanKind? Remove resources require the type to be CLIENT

https://github.com/open-telemetry/opentelemetry-go/blob/fe363be3994012a06e58c713ed0f6e60a2193151/sdk/trace/trace_test.go#L1181

@bonclay7
Copy link
Member Author

Hi @anuraaga, Thanks for the tips, that actually did work !

	tracer := otel.GetTracerProvider().Tracer("petlistadoptions")
	_, span := tracer.Start(ctx, "mssql query", trace.WithSpanKind(trace.SpanKindClient))

The trace appears as remote. I suppose for now there's no detection for the service icon right ?

image

@anuraaga
Copy link
Contributor

Great to hear that helped. Icon and other metadata are populated based on attributes in the span. Database spans would have some of these

https://www.github.com/open-telemetry/opentelemetry-specification/tree/main/specification%2Ftrace%2Fsemantic_conventions%2Fdatabase.md

I guess it's tedious to fill all these yourself though, I hope there is sql instrumentation in go implemented soon so you can just use that. Then it should get recognized properly.

@bhautikpip
Copy link
Contributor

looks like issue is resolved so closing this. Feel free to open if the issue still exists.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants