(core): (asset-staging) Avoid docker bind mounts in asset staging / bundling to allow for more use cases #23393
Closed
1 of 2 tasks
Labels
@aws-cdk/core
Related to core CDK functionality
feature-request
A feature should be added or improved.
needs-triage
This issue or PR still needs to be triaged.
Describe the feature
Docker based bundling should optionally allow for other options than bind mounts for passing the input and output data for bundling to the container.
Use Case
My overall use case is bundling of (python) lambda function within a CICD environment which is within a container, and only has access to the
/var/run/docker.sock
mounted in from the host system.I have worked on the PRs #23330 #23318 and #22829 to resolve this issue, by allowing to mount in the volume from the container running CDK into the bundling container. Sadly the last bit seems not work, due to the bind mounts being dependent on host paths.
https://github.com/aws/aws-cdk/blob/main/packages/%40aws-cdk/core/lib/asset-staging.ts#L436
Proposed Solution
There is different options, which of none I think are a great fit, so I'm happy to hear other ideas.
Option A
Pass environment variables with
sourcePath
andbundleDir
into the container, so the user is able manually make these files available, eg by mounting the outer Volume and providing symlinks.Option B
Offer an alternative variant where a
docker cp
is run before to populate the input dir, and again at the end to copy out the data from the output dir.Docker copy is usually not very fast and performant, but would even work with remote sockets on other hosts.
Option C
Somehow prepend the bind mount path with the correct path to the volume from the outer container
Other Information
The code that creates the bind mount is at https://github.com/aws/aws-cdk/blob/main/packages/%40aws-cdk/core/lib/asset-staging.ts#L435
The directories
/asset-input
and/asset-output
are visible as empty within the bundling container. This comes from the "wrong" bind mount (see below).Output of
docker inspect
of theMounts
section of the bundling container. It shows that thevolumesFrom
mounting of/builds
works correctly (check source path), but the paths for the bind mounts are wrong (would need the volume path as a prefix).Acknowledgements
CDK version used
2.55.1
Environment details (OS name and version, etc.)
Amazon Linux 2
The text was updated successfully, but these errors were encountered: