Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

(aws-lambda-nodejs): AWS SDK v2 is not available for node18.x runtime #22976

Closed
gkkachi opened this issue Nov 18, 2022 · 5 comments · Fixed by #22989
Closed

(aws-lambda-nodejs): AWS SDK v2 is not available for node18.x runtime #22976

gkkachi opened this issue Nov 18, 2022 · 5 comments · Fixed by #22989
Assignees
@corymhall
Labels
@aws-cdk/aws-lambda-nodejs bug This issue is a bug. effort/medium Medium work item – several days of effort p1

Comments

@gkkachi
Copy link
Contributor

gkkachi commented Nov 18, 2022

Describe the bug

According to this comment, node18.x image includes the AWS SDK not v2 but v3.
The AWS SDK v2 is not included in bundles as aws-sdk is specified in externalModules by default, so sccripts which use aws-sdk will fail when using node18.x runtime.

Expected Behavior

extenalModules for node18.x runtime should be @aws-sdk/* .

Current Behavior

{
  "errorType": "Error",
  "errorMessage": "Cannot find module 'aws-sdk'\nRequire stack:\n- /var/task/index.js\n- /var/runtime/index.mjs",
  "trace": [
    "Error: Cannot find module 'aws-sdk'",
    "Require stack:",
    "- /var/task/index.js",
    "- /var/runtime/index.mjs",
    "    at Module._resolveFilename (node:internal/modules/cjs/loader:995:15)",
    "    at Module._load (node:internal/modules/cjs/loader:841:27)",
    "    at Module.require (node:internal/modules/cjs/loader:1061:19)",
    "    at require (node:internal/modules/cjs/helpers:103:18)",
    "    at Runtime.handler (/var/task/index.js:28:3)",
    "    at Runtime.handleOnceNonStreaming (file:///var/runtime/index.mjs:1089:29)"
  ]
}

Reproduction Steps

Deploy a node18.x lambda function whose handler contains require('aws-sdk') and run it.

Possible Solution

Edit here:

aws-cdk/packages/@aws-cdk/aws-lambda-nodejs/lib/bundling.ts

Line 116 in 2c61405

...props.externalModules ?? ['aws-sdk'], // Mark aws-sdk as external by default (available in the runtime)

Additional Information/Context

No response

CDK CLI Version

2.51.0

Framework Version

No response

Node.js Version

v16.18.0

OS

Ubuntu 22.04 LTS

Language

Typescript

Language Version

No response

Other information

No response
@gkkachi gkkachi added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Nov 18, 2022
@trivikr
Copy link
Member

trivikr commented Nov 18, 2022

The fix in CDK should be to ignore all @aws-sdk/* dependencies when Node.js version is 18, and ignore aws-sdk dependency for prior Node.js versions, as pointed out in possible solution.

aws-cdk/packages/@aws-cdk/aws-lambda-nodejs/lib/bundling.ts

Line 116 in 2c61405

...props.externalModules ?? ['aws-sdk'], // Mark aws-sdk as external by default (available in the runtime)

@peterwoodworth peterwoodworth added p1 effort/large Large work item – several weeks of effort and removed needs-triage This issue or PR still needs to be triaged. labels Nov 18, 2022
@yamatatsu
Copy link
Contributor

I try to fix it #22989 .

And workaround of this issue is setting empty array to extenalModules.

@yamatatsu yamatatsu mentioned this issue Nov 21, 2022
Merged
4 tasks
@mrgrain mrgrain added effort/medium Medium work item – several days of effort and removed effort/large Large work item – several weeks of effort labels Nov 25, 2022
@mergify mergify bot closed this as completed in #22989 Nov 25, 2022
mergify bot pushed a commit that referenced this issue Nov 25, 2022
@yamatatsu
fix(lambda-nodejs): AWS SDK v2 is not available for node18.x runtime (#…
55bca78 
…22989)

This PR set default of `externalModules` aws-sdk v3 when use greater than or equal nodejs v18 runtime, because [Lambda runtime nodejs v18 includes aws-sdk v3](https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html#:~:text=Node.js%2018,3.188.0).

fix #22976

----

### All Submissions:

* [x] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md)

### Adding new Unconventional Dependencies:

* [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md/#adding-new-unconventional-dependencies)

### New Features

* [ ] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/main/INTEGRATION_TESTS.md)?
	* [ ] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)?

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@github-actions
Copy link

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

@LarsFronius LarsFronius mentioned this issue Dec 21, 2022
Merged
mergify bot pushed a commit that referenced this issue Jan 20, 2023
@LarsFronius
fix(lambda-nodejs): aws-sdk version detection broken for self-defined…
8a7dffd 
… runtimes (#23416)

We have single-sourced our node versions in our repositories into a single `.nvmrc` at the root of the repo.
From there, we create the Runtimes, like so:
```typescript
const nvmrc = readFileSync(path.join(__dirname, '..', '.nvmrc')).toString().trim()
new Runtime(`nodejs${nvmrc}.x'`, RuntimeFamily.NODEJS, { supportsInlineCode: true })
```
As of #22989  the detection for the appropriate command line flags that dictate which aws-sdk is considered as an external module in esbuild broke for our self-defined Lambda runtimes < node18, as the [codepath](https://github.com/aws/aws-cdk/pull/22989/files#diff-cd86fbd4f2bbefcbcffc2143adccabafa1debe5981edbcdfcc766b5a705fe770R371-R383) detecting runtime versions that ship with the aws-sdk v2 never returned a truthy result.

This left us with Lambdas running node 16, that did not bundle the aws-sdk v3, as it was incorrectly set as external in the esbuild command, which should only be the case from node 18 onwards.
I am thus suggesting explicitly comparing the runtimes family and name to detect whether the runtime is one shipping the aws-sdk v2, as none of the other features of the Runtime class actually have an effect on the Lambda runtime environment.

Related issue: #22976
@Nnaga1 Nnaga1 mentioned this issue Jan 24, 2023
Merged
3 tasks
@juztinlazaro
Copy link

Hi @yamatatsu whats the fix for this issue?

@yamatatsu
Copy link
Contributor

Hi @juztinlazaro
I've fixed it so that nodejs 18 and above will not bundle sdkv3 instead of sdkv2. Also, nodejs 16 and below will not bundle sdkv2 as before.
https://github.com/aws/aws-cdk/pull/22989/files#diff-cd86fbd4f2bbefcbcffc2143adccabafa1debe5981edbcdfcc766b5a705fe770

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@corymhall corymhall

Labels
@aws-cdk/aws-lambda-nodejs bug This issue is a bug. effort/medium Medium work item – several days of effort p1
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(lambda-nodejs): AWS SDK v2 is not available for node18.x runtime yamatatsu/aws-cdk
7 participants
@mrgrain @juztinlazaro @yamatatsu @trivikr @gkkachi @corymhall @peterwoodworth