aws-cdk-lib: AwsCustomResource sometimes does not install dependencies correctly

[ijxy]

Describe the bug

We use a custom resource to invoke a cloudfront invalidation, leveraging the AwsCustomResource construct to do so.

Usually this works fine, but sometimes it fails due to a missing @aws-sdk/client-cloudfront error. In other words, there sometimes seems to be an issue with installing the AWS SDK.

Expected Behavior

The custom resource should install the required AWS SDK dependencies and Just Work.

Current Behavior

Usually the invalidation works as expected, but it regularly fails with the below error:

❌ Deployment failed: Error: The stack named <redacted> failed to deploy: UPDATE_ROLLBACK_COMPLETE: Received response status [FAILED] from custom resource. Message returned: Package @aws-sdk/client-cloudfront does not exist. (RequestId: 864c37-7ce-4ba9-9bdb-9fa82af627)
at FullCloudFormationDeployment.monitorDeployment (/home/runner/work/<redacted>/<redacted>/node_modules/aws-cdk/lib/index.js:426:0236)
at process.processTicksAndRejections (node:internal/process/task_queues:9:)
at async Object.deployStack2 [as deployStack] (/home/runner/work/<redacted>/<redacted>/node_modules/aws-cdk/lib/index.js:429:***3307)
at async /home/runner/work/<redacted>/<redacted>/node_modules/aws-cdk/lib/index.js:429:3698

Reproduction Steps

Below is the exact code for the custom resource that we are using.

import {aws_cloudfront, custom_resources} from "aws-cdk-lib";
import {Construct} from "constructs";

export type CloudFrontInvalidationProps = {
  distribution: aws_cloudfront.IDistribution;
  paths: Array<string>;
};

export class CloudFrontInvalidation extends custom_resources.AwsCustomResource {
  constructor(scope: Construct, {distribution, paths}: CloudFrontInvalidationProps) {
    const id = `CloudFrontInvalidation-${Date.now()}`;
    super(scope, id, {
      onCreate: {
        physicalResourceId: custom_resources.PhysicalResourceId.fromResponse("Invalidation.Id"),
        service: "CloudFront",
        action: "createInvalidation",
        parameters: {
          DistributionId: distribution.distributionId,
          InvalidationBatch: {
            CallerReference: Date.now().toString(),
            Paths: {
              Quantity: paths.length,
              Items: paths,
            },
          },
        },
      },
      policy: custom_resources.AwsCustomResourcePolicy.fromSdkCalls({
        resources: custom_resources.AwsCustomResourcePolicy.ANY_RESOURCE,
      }),
    });
  }
}

We use this construct in conjunction with two BucketDeployments such that the invalidation is triggered after the distribution and bucket deployments have finished updating, like so:

const invalidation = new CloudFrontInvalidation(this, {
  distribution,
  paths: ["/*"],
});
invalidation.node.addDependency(distribution, hashedAssetsDeployment, unhashedAssetsDeployment);

Possible Solution

https://github.com/aws/aws-cdk/tree/main/packages/aws-cdk-lib/custom-resources

The manner in which AWS SDK dependencies are handled seems to be by installing the V2 SDK and then using an adapter.

I don't see why it needs to be so convoluted and brittle–at this point in time, using the V3 SDK directly seems like the sensible thing to do since they are pre-installed in the Lambda Runtime on Node ≥18 (ie installing them should not even be needed).

Additional Information/Context

No response

CDK CLI Version

2.90.0 (build 8c535e4)

Framework Version

No response

Node.js Version

18

OS

Linux

Language

TypeScript

Language Version

No response

Other information

No response

[khushail]

Hi @jxdp ,thanks for reaching out. It looks like you are using an older version of CDK, The latest version supports Lambda Node 18 runtime and SDK V3 and here are few PRs merged in the latest releases -

#27066
#26710
#25406
#26212

Could you try using the latest version and see if that addresses the issue you have mentioned.

[github-actions]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.