feat(core): improved docker bundling performance on mac os

[Dzhuneyt]

First time contributor to AWS CDK here. Let me know if I've missed something.

As a developer who uses macOS for development purposes on a daily basis and AWS CDK for an enterprise project, I am really impacted by the official and well known Docker volumes performance issue. Since the project that we are working on, relies heavily on Lambdas that are primarly written in TypeScript, the mass deployment of these Lambdas using AWS CDK CLI and the (recently introduced) NodejsFunction construct takes a significant amount of time and causes a CPU overload.

This is mainly due to the fact that the construct internally does the following things:

1. Starting a docker container with Parcel
2. Mounting the root of the closest folder that contains .git (this is the root of the whole CDK app for us, that includes the CDK stacks, the Lambdas and any other utility functions and node_modules folders, since we opted for the "monorepo" structural pattern)
3. Runs the parcel bundler inside the container, which emits a single .js file as an asset that will serve as the Lambda's "code"
4. Deploys the .js file using the standard lambda.Function construct (NodejsFunction is just a wrapper of lambda.Function with @aws-cdk/core bundling and Parcel involved).

For our project, this means CDK deployment spawns dozens of containers (one per Lambda), that all mount (using Docker volumes), the whole project. The frequent IO operations by Parcel within the container cause a massive CPU spike in all of these containers, causing each Lambda compilation to take 1-2 seconds. This latency is quickly magnified as the number of Lambdas grow within the project.

The latency and CPU spike is mainly a side effect of how Docker and volumes work in macOS for which you can read more on the above link. This is a common pain point for all developers who use these tools, even outside the AWS CDK world. Here are some examples and further reading, including further reading on how flags like ":cached" or ":delegated" help:

• High CPU Utilization of Hyperkit in Mac docker/for-mac#1759 (comment)
• https://stackoverflow.com/questions/58277794/diagnosing-high-cpu-usage-on-docker-for-mac/58293240#58293240
• https://stackoverflow.com/questions/51694789/macbook-pro-2018-high-temperature-and-cpu-usage-with-docker/51698665#51698665
• https://stackoverflow.com/questions/60878918/docker-hyperkit-process-cpu-usage-going-crazy-how-to-keep-it-under-control#comment107712547_60878918
• https://stackoverflow.com/questions/55951014/docker-in-macos-is-very-slow/55953023#55953023

I've decided to use ":delegated" here since the bundling mechanism generates files inside the container that need replication on the host machine with small tolerable delay, not the other way around (where ":cached" would have been useful). The addition of the flag is non-conditional based on the current OS, because the existence of the flag is a NO-OP in all other operating systems (tested) and is only taken into account by Docker when the volume driver is detected to be osxfs.

Closes #8544

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

[Dzhuneyt]

@jogold @eladb ready for another round of review.

[jogold]

@Dzhuneyt from the Build Logs

@aws-cdk/core: error: [awslint:docs-public-apis:@aws-cdk/core.DockerConsistency.DEFAULT] Public API element must have a docstring

DockerConsistency.DEFAULT is not "jsdocumented". But I suggest dropping this DEFAULT and use DELEGATED directly instead. How about renaming the enum to DockerVolumeConsistency?

[Dzhuneyt]

@jogold addressed both your concerns. Although honestly I see benefit in leaving the DEFAULT option as an "alias" to another option for futureproofing and giving the option for the framework to make changes to what is the default value without introducing breaking changes.

[jogold]

@eladb LGTM

[aws-cdk-automation]

AWS CodeBuild CI Report

• CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
• Commit ID: 9d392a8
• Result: SUCCEEDED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[mergify]

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).