SAM build doubles the size of package when having more than 1 function per directory/project for node.js builds

[lucasam]

Issue description:

Given that:

• template.yaml and package.json at the root directory of the node aplicaiton.
• More than 1 lambda function declared at template.yaml.

Issue:

Altough sam build completes sucessfully, the second function has double the size of the first one.
This behaviors grows exponentially (third function will double the second and so on).
The problem here is that the folder of the second function contains .aws-sam of the first one

.aws-sam/build/FunctionOne
.aws-sam/build/FunctionTwo/
.aws-sam/build/FunctionTwo/.aws-sam/build/FunctionOne

Sample code can be found here https://github.com/lucasam/sambuildissue1

Reproducing the issue

git clone https://github.com/lucasam/sambuildissue1
cd sambuildissue1

Then build

lucasam$ sam build --debug
Using SAM Template at /Users/lucasam/Documents/src/sambuildissue1/.aws-sam/build/FunctionTwo/template.yaml
Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
'build' command is called
No Parameters detected in the template
2 resources found in the template
Found Serverless function with name='FunctionOne' and CodeUri='.'
Found Serverless function with name='FunctionTwo' and CodeUri='.'
Building resource 'FunctionOne'
Loading workflow module 'aws_lambda_builders.workflows'
Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)'
Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'
Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)'
Registering workflow 'GoDepBuilder' with capability 'Capability(language='go', dependency_manager='dep', application_framework=None)'
Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)'
Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)'
Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)'
Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)'
Found workflow 'NodejsNpmBuilder' to support capabilities 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'
Running workflow 'NodejsNpmBuilder'
Running NodejsNpmBuilder:NpmPack
NODEJS packaging file:/Users/lucasam/Documents/src/sambuildissue1/.aws-sam/build/FunctionTwo to /var/folders/sj/60s2bzdn7c52vwbq64d8tr9d64r349/T/tmph7xegid2
executing NPM: ['npm', 'pack', '-q', 'file:/Users/lucasam/Documents/src/sambuildissue1/.aws-sam/build/FunctionTwo']
NODEJS packed to build-issue-0.0.1.tgz
NODEJS extracting to /var/folders/sj/60s2bzdn7c52vwbq64d8tr9d64r349/T/tmph7xegid2/unpacked
NodejsNpmBuilder:NpmPack succeeded
Running NodejsNpmBuilder:CopyNpmrc
NodejsNpmBuilder:CopyNpmrc succeeded
Running NodejsNpmBuilder:CopySource
NodejsNpmBuilder:CopySource succeeded
Running NodejsNpmBuilder:NpmInstall
NODEJS installing in: /Users/lucasam/Documents/src/sambuildissue1/.aws-sam/build/FunctionTwo/.aws-sam/build/FunctionOne
executing NPM: ['npm', 'install', '-q', '--no-audit', '--no-save', '--production']
NodejsNpmBuilder:NpmInstall succeeded
Running NodejsNpmBuilder:CleanUpNpmrc
NodejsNpmBuilder:CleanUpNpmrc succeeded
Building resource 'FunctionTwo'
Loading workflow module 'aws_lambda_builders.workflows'
Found workflow 'NodejsNpmBuilder' to support capabilities 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'
Running workflow 'NodejsNpmBuilder'
Running NodejsNpmBuilder:NpmPack
NODEJS packaging file:/Users/lucasam/Documents/src/sambuildissue1/.aws-sam/build/FunctionTwo to /var/folders/sj/60s2bzdn7c52vwbq64d8tr9d64r349/T/tmpzvnf85j5
executing NPM: ['npm', 'pack', '-q', 'file:/Users/lucasam/Documents/src/sambuildissue1/.aws-sam/build/FunctionTwo']
NODEJS packed to build-issue-0.0.1.tgz
NODEJS extracting to /var/folders/sj/60s2bzdn7c52vwbq64d8tr9d64r349/T/tmpzvnf85j5/unpacked
NodejsNpmBuilder:NpmPack succeeded
Running NodejsNpmBuilder:CopyNpmrc
NodejsNpmBuilder:CopyNpmrc succeeded
Running NodejsNpmBuilder:CopySource
NodejsNpmBuilder:CopySource succeeded
Running NodejsNpmBuilder:NpmInstall
NODEJS installing in: /Users/lucasam/Documents/src/sambuildissue1/.aws-sam/build/FunctionTwo/.aws-sam/build/FunctionTwo
executing NPM: ['npm', 'install', '-q', '--no-audit', '--no-save', '--production']
NodejsNpmBuilder:NpmInstall succeeded
Running NodejsNpmBuilder:CleanUpNpmrc
NodejsNpmBuilder:CleanUpNpmrc succeeded

Build Succeeded

Built Artifacts  : .aws-sam/build
Built Template   : .aws-sam/build/template.yaml

Commands you can use next
=========================
[*] Invoke Function: sam local invoke
[*] Package: sam package --s3-bucket <yourbucket>
    
Sending Telemetry: {'metrics': [{'commandRun': {'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '', 'commandName': 'sam build', 'duration': 10815, 'exitReason': 'success', 'exitCode': 0, 'requestId': '794b5afe-5db2-4c2e-94f1-46c908e8a0ad', 'installationId': '893c5aa8-d107-4b3b-8f82-e7cb5c9ce2fe', 'sessionId': 'bb97b105-48f6-4dde-92ac-4b421ec6aca1', 'executionEnvironment': 'CLI', 'pyversion': '3.7.4', 'samcliVersion': '0.22.0'}}]}
HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1)

When you inspect generated files the output of the first sam build is packaged at the second function

lucasam$ cd .aws-sam/build/

lucasam$ du -hs *
 43M    FunctionOne
 86M    FunctionTwo               // SAME CODEBASE, DOUBLE THE SIZE
4.0K    template.yaml

lucasam$ cd FunctionTwo/
lucasam$ ls -lia
total 16
34492672 drwxr-xr-x   7 lucasam  ANT\Domain Users  224 Oct 17 10:31 .
34489527 drwxr-xr-x   5 lucasam  ANT\Domain Users  160 Oct 17 10:31 ..
34492678 drwxr-xr-x   3 lucasam  ANT\Domain Users   96 Oct 17 10:31 .aws-sam     // THIS FOLDER SHOULD NOT BE HERE
34492675 drwxr-xr-x   4 lucasam  ANT\Domain Users  128 Oct 17 10:31 lib
34494271 drwxr-xr-x  17 lucasam  ANT\Domain Users  544 Oct 17 10:31 node_modules
34492674 -rw-r--r--   1 lucasam  ANT\Domain Users  291 Oct 26  1985 package.json
34492673 -rw-r--r--   1 lucasam  ANT\Domain Users  469 Oct 26  1985 template.yaml

Observe that:

• FunctionTwo contains .aws-sam directory built during functionOne build.
• The size of FunctionTwo package is the double of the first one.
• Similiar behavior does not apply to java projects
• If you move template.yaml to parent directory, the behavior changes to the proper one

Note: #1236 / #1180 might fix this issue.

[jfuss]

@lucasam Thanks for the report. I don't think any of those issues are related here but seems exactly the same as aws/aws-lambda-builders#130 but with a different workflow. The Node builder has the same Excludes that is mentioned in the PR I linked: https://github.com/awslabs/aws-lambda-builders/blob/develop/aws_lambda_builders/workflows/nodejs_npm/workflow.py#L24. I haven't tested it but this could be the cause.

There is another way to solve this as well. Part of the reason this is coming up is because your CodeUri is the cwd. This tells sam build that the cwd is all of your source code for each function (including template, tests, and anything else in your top directory). We view CodeUri as a place for all your functions code and file that need to run, you shouldn't need tests or the template in the zip on lambda. The reason .aws-sam is appearing in the second function is because it runs after the first one is built and therefore the .aws-sam directory exists. If instead you change the CodeUri to CodeUri: lib and then adjust your handler files. You can then run sam build -m package.json to build off the package.json file in the root of your project (cwd). We consider having CodeUri as cwd an anti pattern since it will include any and all files. Again I do think there is a bug here with the .aws-sam getting copied over but there is only so much generic excluding we can do here.

[jfuss]

@lucasam Thank you for providing a full working example. I change that line I linked about to EXCLUDED_FILES = (".aws-sam", ".git") and installed the CLI and Lambda Builders locally. I do not see the issue anymore (see output below). I am going to work on the fix in Lambda Builders for this.

(aws-sam-cli) ➜  sambuildissue1 git:(master) samdev build
Building resource 'FunctionOne'
Running NodejsNpmBuilder:NpmPack
Running NodejsNpmBuilder:CopyNpmrc
Running NodejsNpmBuilder:CopySource
Running NodejsNpmBuilder:NpmInstall
Running NodejsNpmBuilder:CleanUpNpmrc
Building resource 'FunctionTwo'
Running NodejsNpmBuilder:NpmPack
Running NodejsNpmBuilder:CopyNpmrc
Running NodejsNpmBuilder:CopySource
Running NodejsNpmBuilder:NpmInstall
Running NodejsNpmBuilder:CleanUpNpmrc

Build Succeeded

Built Artifacts  : .aws-sam/build
Built Template   : .aws-sam/build/template.yaml

Commands you can use next
=========================
[*] Invoke Function: sam local invoke
[*] Package: sam package --s3-bucket <yourbucket>

(aws-sam-cli) ➜  sambuildissue1 git:(master) cd .aws-sam/build/
(aws-sam-cli) ➜  build git:(master) ls
FunctionOne   FunctionTwo   template.yaml
(aws-sam-cli) ➜  build git:(master) du -hs *
 43M	FunctionOne
 43M	FunctionTwo
4.0K	template.yaml
(aws-sam-cli) ➜  build git:(master) cd FunctionTwo/
(aws-sam-cli) ➜  FunctionTwo git:(master) ls -lia
total 32
3248501 drwxr-xr-x   7 <>  <> Users   224 Oct 18 09:45 .
3245547 drwxr-xr-x   5 <>  <> Users   160 Oct 18 09:45 ..
3248503 -rw-r--r--   1 <>  <> Users  6530 Oct 26  1985 README.md
3248505 drwxr-xr-x   4 <>  <> Users   128 Oct 18 09:45 lib
3248609 drwxr-xr-x  17 <>  <> Users   544 Oct 18 09:45 node_modules
3248504 -rw-r--r--   1 <>  <> Users   291 Oct 26  1985 package.json
3248502 -rw-r--r--   1 <>  <> Users   469 Oct 26  1985 template.yaml

[naphtul]

A similar issue happens with Layers.
Please see reproduction of this issue here:
https://github.com/naphtul/samNodeLayer/tree/master/nodejs
Run sam build --debug (which runs successfully).
Then run sam package --debug --s3-bucket <BUCKET_NAME> --output-template-file packaged.yaml
This creates two output files: One is double the size of the other due to the .aws-sam folder.

My bigger issue is that I can't find a folder structure from an official template example of a NodeJS node_modules folder as a Layer, while the source code footprint remains small (i.e. without node_modules).

[jfuss]

The fix was released in v0.8.0 of aws-lambda-builders and then released in [v0.44.0](https://github.com/awslabs/aws-sam-cli/releases/tag/v0.44.0 of SAM CLI.

Closing as this has been fixed.