AWS SAM & AWS Lambda: pg spans not reported #13975
Comments
github-actions
bot
added
the
Package: aws-serverless
|
Hi @ffraenz, thank you for filing this and providing comprehensive reproduction. I ran through your repo and can confirm the same outcome. We're going to investigate this further, it might be related to drizzle - we've had problems with it in the past, although those were related to ESM and opentelemetry wrapping. |
|
I've added a second test case to the repository, using Sentry with pg, leaving out Drizzle.js. I get the same unexpected outcome. It doesn't seem to be related to Drizzle 🤔
function-pg logs from AWS CloudWatch |
|
@ffraenz thanks for the second test case, that'll save time investigating drizzle :). From the logs it's already weird that we're starting a second root span. That indicates that the lambda handler was not wrapped by otel correctly. sentry-javascript/packages/aws-serverless/src/sdk.ts Lines 333 to 356 in 41515a6 I'll try to investigate that further. Thanks again for providing great reproduction! |
|
I just tested your pg handler code in an node app and indeed everything was instrumented fine there, so it's probably not an issue with our pg instrumentation either. Probably an issue with the aws-serverless sdk. |
|
@ffraenz we've figured out why there are no pg spans. Since you're using SAM with esbuild to bundle the lambda function, all the dependencies are bundled in as well, including Therefore, to get From my testing, there are still
|
|
I was not aware of the way opentelemetry hooks into pg via module requires/imports that are usually replaced with references to in-memory versions of the modules during esbuild bundling. I see that esbuild plugins for this use case are already being discussed: open-telemetry/opentelemetry-js#4174. I try my best to avoid the added complexity of maintaining Lambda layers. Making pg external and providing it via a Lambda layer may lead to version mismatches. So, I'm wondering if manual spans might be the best way to deal with it in my code base. All SQL queries are already executed in a wrapper function of mine, which I could extend with such instrumentation logic. I can't assess if that solution could come close to what opentelemetry pg hooks have to offer, though, in addition to Sentry post processing. As AWS SAM is one of the recommended ways to ship Node.js code on AWS Lambda, I think it makes sense to mention this limitation in the Sentry docs. |
|
@ffraenz that's a good suggestion. As for Lambda Layer, yea I can understand. You can have a look at what opteneltemetry instruments by looking at their tests. Have a look at our docs for custom instrumentation to understand what sentry span apis you might want to use. Lastly, I agree we should have some notes about bundled setups. I'l add it our backlog. |
Is there an existing issue for this?
How do you use Sentry?
Sentry Saas (sentry.io)
Which SDK are you using?
@sentry/aws-serverless
SDK Version
8.34.0
Framework Version
SAM CLI 1.125.0, pg 8.13.0, Drizzle ORM 0.34.1
Link to Sentry event
https://wierk.sentry.io/performance/trace/fd08bbde979368bd0e7e26c950fcdd92/?node=ms-6ae4ec7ba851d454&node=txn-dc57263b08454d93b3a3001bda12236e&pageEnd&pageStart&project=4508123298463824&source=traces&statsPeriod=24h×tamp=1728945399.927
Reproduction Example/SDK Setup
I'd like to configure error reporting and profiling via @sentry/aws-serverless for a project that builds on the AWS SAM framework. AWS SAM offers tooling to build and deploy TypeScript source code to a Node.js AWS Lambda function via CloudFormation. The project uses Drizzle ORM and pg to interact with a database.
I've created a repository containing a minimal reproducible example: https://github.com/ffraenz/aws-sam-ts-pg-sentry.
Steps to Reproduce
Configure, build, and deploy the minimal reproducible example above on AWS and trigger it by clicking "Test" on the AWS Lambda UI in AWS Console. Results show up in the Sentry UI.
Expected Result
The function runs successfully. A Sentry transaction gets reported containing a trace with spans for two HTTP events and two Postgres queries executed via Drizzle.js and pg.
Actual Result
The function runs successfully. A Sentry transaction gets reported containing a trace with spans for two HTTP events but without any spans for pg queries. The missing span explicitly shows as "Missing instrumentation" in the UI. The trace shown in the following screenshot is linked above.
For reference, this is the
Sentry.initcall from the repository mentioned above:AWS Lambda logs from AWS CloudWatch
The text was updated successfully, but these errors were encountered: