Running sam sync with esbuild does not include dependencies

[snorreks]

Description:

Running sam sync with esbuild does not include the dependencies.
Running sam deploy works.

Steps to reproduce:

1. Include a dependency:

{
	"name": "hello_world",
	"main": "app.js",
	"dependencies": {
		"uuidv4": "^6.2.12"
	},
...

2. Include it in your app.ts

import type { APIGatewayProxyHandler } from 'aws-lambda';
import { uuid } from 'uuidv4';

export const lambdaHandler: APIGatewayProxyHandler = async (event) => {
	try {
		console.log('event', event);
		return {
			body: JSON.stringify({
				message: 'hello ' + uuid() + process.env.FLAVOR,
			}),
			statusCode: 200,
		};
	} catch (err) {
		console.log(err);
		return {
			body: JSON.stringify({
				message: 'some error happened',
			}),
			statusCode: 500,
		};
	}
};

3. Use esbuild as the BuildMethod:

    HelloWorldFunction:
        Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
        Properties:
            CodeUri: hello/
            Handler: app.lambdaHandler
            Runtime: nodejs14.x
            Architectures:
                - x86_64
            Environment:
                Variables:
                    FLAVOR: !Ref FLAVOR
            Events:
                HelloWorld:
                    Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
                    Properties:
                        Path: /hello
                        Method: get
        Metadata: # Manage esbuild properties
            BuildMethod: esbuild
            BuildProperties:
                Minify: true
                Target: 'es2020'
                Sourcemap: true
                EntryPoints:
                    - app.ts

4. Run:

sam sync --stack-name ... --watch

Observed result:

There is no error in the terminal:

2022-03-03 16:05:33 - Waiting for stack create/update to complete

CloudFormation events from stack operations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------    
ResourceStatus                                       ResourceType                                         LogicalResourceId                                    ResourceStatusReason
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------    
UPDATE_IN_PROGRESS                                   AWS::CloudFormation::Stack                           test-app                                             Transformation succeeded
CREATE_IN_PROGRESS                                   AWS::CloudFormation::Stack                           AwsSamAutoDependencyLayerNestedStack                 -
CREATE_IN_PROGRESS                                   AWS::CloudFormation::Stack                           AwsSamAutoDependencyLayerNestedStack                 Resource creation Initiated
CREATE_COMPLETE                                      AWS::CloudFormation::Stack                           AwsSamAutoDependencyLayerNestedStack                 -
UPDATE_IN_PROGRESS                                   AWS::Lambda::Function                                HelloWorldFunction                                   -
UPDATE_COMPLETE                                      AWS::Lambda::Function                                HelloWorldFunction                                   -
UPDATE_COMPLETE                                      AWS::CloudFormation::Stack                           test-app                                             -
UPDATE_COMPLETE_CLEANUP_IN_PROGRESS                  AWS::CloudFormation::Stack                           test-app                                             -
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

CloudFormation outputs from deployed stack
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Outputs
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------   
Key                 HelloWorldFunctionIamRole
Description         Implicit IAM Role created for Hello World function
Value               arn:aws:iam::102512246328:role/test-app-HelloWorldFunctionRole-IFB6LCDCZFB8

Key                 HelloWorldApi
Description         API Gateway endpoint URL for Prod stage for Hello World function
Value               https://xfi7je9i27.execute-api.eu-north-1.amazonaws.com/Prod/hello/

Key                 HelloWorldFunction
Description         Hello World Lambda Function ARN
Value               arn:aws:lambda:eu-north-1:102512246328:function:test-app-HelloWorldFunction-mJ9kipF6iwEi
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------   

Stack update succeeded. Sync infra completed.

{'StackId': 'arn:aws:cloudformation:eu-north-1:102512246328:stack/test-app/0c378cc0-9960-11ec-9d59-0e1e6bf4f0ce', 'ResponseMetadata': {'RequestId': '041daa8a-5770-4086-b8d2-3ddf51d2c7f9', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '041daa8a-5770-4086-b8d2-3ddf51d2c7f9', 'content-type': 'text/xml', 'content-length': '379', 'date': 'Thu, 03 Mar 2022 15:05:29 GMT'}, 'RetryAttempts': 0}}
Infra sync completed.

but in cloud watch i get this:

{
    "errorType": "Runtime.ImportModuleError",
    "errorMessage": "Error: Cannot find module 'uuidv4'\nRequire stack:\n- /var/task/app.js\n- /var/runtime/UserFunction.js\n- /var/runtime/index.js",
    "stack": [
        "Runtime.ImportModuleError: Error: Cannot find module 'uuidv4'",
        "Require stack:",
        "- /var/task/app.js",
        "- /var/runtime/UserFunction.js",
        "- /var/runtime/index.js",
        "    at _loadUserApp (/var/runtime/UserFunction.js:202:13)",
        "    at Object.module.exports.load (/var/runtime/UserFunction.js:242:17)",
        "    at Object.<anonymous> (/var/runtime/index.js:43:30)",
        "    at Module._compile (internal/modules/cjs/loader.js:1085:14)",
        "    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)",
        "    at Module.load (internal/modules/cjs/loader.js:950:32)",
        "    at Function.Module._load (internal/modules/cjs/loader.js:790:12)",
        "    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:76:12)",
        "    at internal/main/run_main_module.js:17:47"
    ]
}

Expected result:

Have the lambda function not crashing when using sam sync.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

1. OS: Windows x64
2. sam --version: 1.40.1
3. AWS region: eu-north-1

[mildaniel]

Hi @snorreks, thanks for bringing this to our attention. When using sync, sam will attempt to package the dependencies in a separate dependency layer, but it seems like it's not finding the package there. I will try to reproduce and look further into it. It would also be helpful if you could provide us the layer zip for the generated auto dependency layer.

In the meantime, try running sync with the --no-dependency-layer flag and see if that works for you.

[richardwu]

Coincidentally I also ran into this issue just now. The --no-dependency-layer flag fixed it for me, however this prevents the editor in the Lambda UI from loading the source file (since it's >3MB for me).

Is there a way to hook into the build process of esbuild here?

[snorreks]

@mildaniel --no-dependency-layer worked!

[mildaniel]

Hey @richardwu. I'm sorry you ran into this. Would you be able to share the layer zip for the sam-generated auto dependency layer? Seems like this is a transient issue that doesn't always occur, so it would be helpful for pin-pointing when it does.

There isn't currently a way for hooking into the build process but we're exploring ways for making this experience better.

[altrr2]

Hope it helps...

For anyone running into the same problem, as an alternative workaround it is possible to use npm workspaces with an additional package.json in the root of the sam app project e.g.

{
    "name": "@hello_world/sam_app",
    "version": "1.0.0",
    "private": true,
    "workspaces": [
        "hello-world/"
    ],
    "devDependencies": {
        "@types/aws-lambda": "^8.10.92",
        "@types/jest": "^27.4.0",
        "@types/node": "^17.0.13",
        "@typescript-eslint/eslint-plugin": "^5.10.2",
        "@typescript-eslint/parser": "^5.10.2",
        "esbuild": "^0.14.14",
        "esbuild-jest": "^0.5.0",
        "eslint": "^8.8.0",
        "eslint-config-prettier": "^8.3.0",
        "eslint-plugin-prettier": "^4.0.0",
        "jest": "^27.5.0",
        "prettier": "^2.5.1",
        "ts-node": "^10.4.0",
        "typescript": "^4.5.5"
    }
}

After running npm install node_modules will be initiated in the root of the sam app project instead of the function code folder, without breaking sam sync (which doesnt seem to work if node_modules exists in the function code folder). Then modules can be installed into workspace using something like npm install axios -w hello-world.

(I also moved most of devDependencies out of the function's package.json keeping only esbuild* there, what seems to speed up the build a bit), so it looks like this:

{
  "name": "@hello_world/hello_world",
  "version": "1.0.0",
  "description": "hello world sample for NodeJS",
  "main": "app.js",
  "repository": "https://github.com/awslabs/aws-sam-cli/tree/develop/samcli/local/init/templates/cookiecutter-aws-sam-hello-nodejs",
  "author": "SAM CLI",
  "license": "MIT",
  "dependencies": {
    "axios": "^0.27.2"
  },
  "scripts": {
    "unit": "jest",
    "lint": "eslint '*.ts' --quiet --fix",
    "compile": "tsc",
    "test": "npm run compile && npm run unit"
  },
  "devDependencies": {
    "esbuild": "^0.14.14",
    "esbuild-jest": "^0.5.0"
  }
}

[sriram-mv]

Closing based on aws/aws-lambda-builders#368