@aws-cdk/aws-servicecatalog Ephemeral storage modification support

[KetulRadadiya]

Describe the feature

Currently, AWS CDK’s @aws-cdk/aws-servicecatalog module uses an internal Lambda function for asset copying during Service Catalog product deployment. This Lambda’s configuration (e.g., ephemeral storage) is not exposed for user customization.

This feature would allow developers to override or specify these Lambda ephemeral storage when deploying Service Catalog products.

Expected Enhancements:

1. Add properties (e.g., lambdaProps) to allow users to configure the internal Lambda's Ephemeral storage (similar to constructs like aws-lambda.Function OR aws-s3-deployment.BucketDeployment).
2. Ensure this configurations is exposed as optional parameters during servicecatalog.ProductStack initialization.

Use Case

When deploying large asset bundles around of 3 GB (e.g., Lambda function packages, website files) to Service Catalog products, the internal Lambda function fails due to insufficient ephemeral storage.

For example:

• "I’m frustrated when large files (>500 MB) cause the internal Lambda function to fail with ENOSPC (no space left on device), and I cannot adjust the ephemeral storage or other limits."
• "Currently, there’s no way to address these issues without extensive custom workarounds that re-implement the CDK logic for asset management."
  This feature would significantly improve flexibility for handling larger workloads in Service Catalog deployments.

Proposed Solution

Add an optional property to the servicecatalog.ProductStackProps that allows customization of the internal Lambda function.

interface ProductStackProps {
  ...
  lambdaProps?: lambda.FunctionProps; // Allow custom configuration
}

Developers can pass lambdaProps to override settings for the internal Lambda, such as:

new servicecatalog.ProductStack(this, 'TestProductStack', {
  assetBucket: userDefinedBucket,
  lambdaProps: {
    memorySize: 1024,
    ephemeralStorageSize: Size.gibibytes(3), // Customize storage
  },
});

Other Information

The error that I'm facing,

download failed: s3://cdk-hnb659fds-assets-1234567899-us-east-2/f38cbf44b36acd518668341f19c50896575a52313f6ea12dcad359f3fccd9978.zip to ../../tmp/tmp44wplx45/contents/f38cbf44b36acd518668341f19c50896575a52313f6ea12dcad359f3fccd9978.zip [Errno 28] No space left on device
[ERROR]	2024-12-02T12:01:14.436Z	5e3f2fc5-6474-4ebr-be6d-e63abb99561f	Command '['/opt/awscli/aws', 's3', 'cp', 's3://cdk-hnb659fds-assets-1234567899-us-east-2/f38cbf44b36acd518668341f19c50896575a52313f6ea12dcad359f3fccd9978.zip', '/tmp/tmp44wplx45/contents']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/var/task/index.py", line 130, in handler
    s3_deploy(s3_source_zips, s3_dest, user_metadata, system_metadata, prune, exclude, include, source_markers, extract)
  File "/var/task/index.py", line 195, in s3_deploy
    aws_command("s3", "cp", s3_source_zip, contents_dir)
  File "/var/task/index.py", line 259, in aws_command
    subprocess.check_call([aws] + list(args))
  File "/var/lang/lib/python3.9/subprocess.py", line 373, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/opt/awscli/aws', 's3', 'cp', 's3://cdk-hnb659fds-assets-1234567899-us-east-2/f38cbf44b36acd518668341f19c50896575a52313f6ea12dcad359f3fccd9978.zip', '/tmp/tmp44wplx45/contents']' returned non-zero exit status 1.

Ref: https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_servicecatalog.ProductStackProps.html

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

CDK version used

2.170.0

Environment details (OS name and version, etc.)

Ubuntu 22.04.5 LTS

[ashishdhingra]

Findings:

• ProductStack uses ProductStackSynthesizer.
• ProductStackSynthesizer uses BucketDeployment here.
• BucketDeployment uses BucketDeploymentSingletonFunction here
  • ephemeralStorageSize is already exposed as an optional property in BucketDeploymentProps.
  • While ProductStackSynthesizer exposes memoryLimit property in ProductStackSynthesizerProps, it doesn't expose ephemeralStorageSize.
  • ProductStack also exposes memoryLimit in ProductStackProps, but doesn't expose ephemeralStorageSize.

So change would involve exposing ephemeralStorageSize in:

• ProductStackSynthesizerProps and use it while performing BucketDeployment.
• ProductStackProps and use it while initializing ProductStackSynthesizer.

@KetulRadadiya Feel free to contribute PR based on above inputs, if possible (also refer https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md to refer requirements of a PR, which includes adding unit tests and updating integration test snapshot).

[yash30389]

Ability to Configure EphemeralStorage for Lambda in ServiceCatalog.ProductStack

Problem Statement:
I use the ServiceCatalog.ProductStack to define resources inside my stack. By default, a Lambda function is created to copy assets from the AWS core bucket to a custom, newly created bucket.

However, I encountered the following challenges:

1. I cannot determine where the Lambda function's source code resides in the node_modules.
2. While I can increase the memory size using the memoryLimit property for ProductStack, there is no property or configuration to enhance the Lambda function's ephemeralStorage.

Use Case:
I want to create many resources inside the ProductStack that require increased ephemeral storage (up to 7629.39 MiB). Unfortunately, the default Lambda function does not provide this capability, which limits the stack's scalability.

Add an optional property, ephemeralStorageLimit, to the ProductStack construct to specify the required size for the Lambda function.

Current Workaround:
Currently, there is no workaround for enhancing ephemeralStorage for this Lambda function within the ProductStack.