Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(custom-resources): fails to use latest SDK version #29958

Merged
merged 5 commits into from
Apr 27, 2024

Conversation

moelasmar
Copy link
Contributor

@moelasmar moelasmar commented Apr 25, 2024

Issue # (if applicable)

Closes #29891

Reason for this change

When setting AwsCustomResource with installLatestAwsSdk: true it fails to upgrade aws-sdk to latest version. The Lambda function created to support the custom resource usually time out after 120 seconds, and for some cases it get time out even after 900 seconds.

Description of changes

Update the Lambda function created for the custom resource to set its MemorySize to be 512 in case if installLatestAwsSdk flag is true instead of the default value 128.

Also, this change will expose the MemorySize to the AwsCustomResource construct, so customers can customize the MemorySize of the Lambda function to fulfill use cases that requires higher memory, CPU, or I/O performance.

Description of how you validated changes

I added the unit test cases, and updated the integration test cases.

Checklist


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@aws-cdk-automation aws-cdk-automation requested a review from a team April 25, 2024 07:06
@github-actions github-actions bot added effort/medium Medium work item – several days of effort p2 labels Apr 25, 2024
@mergify mergify bot added the contribution/core This is a PR that came from AWS. label Apr 25, 2024
Copy link
Collaborator

@aws-cdk-automation aws-cdk-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.

@TheRealAmazonKendra
Copy link
Contributor

I'm not a fan of this change unless we really understand what's going on and why this suddenly started. It seems more like a bandaid than an actual fix.

We need to understand why this issue just started. We didn't see it prior to last week and, from what I know, there were no changes on our end.

Did Lambda do something that is impacting this on their end? Or, is there actually a change we made that is related. Is npm experiencing outages and latency issues? Regardless, we shouldn't just make an update without understanding root cause as we are seeing this issue independent of the cdk version a customer is using.

@colifran was also doing some experiments around this issue on Friday and can confirm that was independent of cdk version.

@TheRealAmazonKendra
Copy link
Contributor

I also commented this is the issue but just to make sure the comment doesn't get lost:

I'm going to suspect there's something going on with the SDK and/or npm here. On https://www.npmjs.com/package/@aws-sdk/client-s3 it says the most recent version is from 9 days ago and lists 3.556.0 as the most up-to-date version. On https://github.com/aws/aws-sdk-js-v3 latest is 3.562.0 published 12 hours ago.

@mrgrain
Copy link
Contributor

mrgrain commented Apr 25, 2024

This is interesting. I tend to agree with you @moelasmar that 128MB memory is not a good choice for this function. Lambda Functions use memory as a proxy configuration for other resources. In particular, with higher memory the Function also receives increased CPU and I/O performance.

CPU performance is extremely relevant to cryptographic code, e.g. TLS connections. In the context of making AWS API requests, this is highly relevant. Not only is it using TLS to connect to APIs, it will also sign requests for authentication. You might find existing literature on this; from personal experience crypto performance starts to plateau around 768MB.

I/O performance will be relevant mostly for npm install because that is going to write a LOT of files to disk.

# Conflicts:
#	packages/@aws-cdk-testing/framework-integ/test/aws-ec2/test/integ.instance-ebs-optimized.js.snapshot/integ-ec2-instance-ebs-optimized.assets.json
#	packages/@aws-cdk-testing/framework-integ/test/custom-resources/test/aws-custom-resource/integ.aws-custom-resource.js.snapshot/aws-cdk-sdk-js-v3.assets.json
#	packages/@aws-cdk-testing/framework-integ/test/custom-resources/test/aws-custom-resource/integ.aws-custom-resource.js.snapshot/aws-cdk-sdk-js.template.json
#	packages/@aws-cdk-testing/framework-integ/test/custom-resources/test/aws-custom-resource/integ.aws-custom-resource.js.snapshot/manifest.json
#	packages/@aws-cdk-testing/framework-integ/test/custom-resources/test/aws-custom-resource/integ.aws-custom-resource.js.snapshot/tree.json
@aws-cdk-automation aws-cdk-automation dismissed their stale review April 27, 2024 01:25

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

@aws-cdk-automation aws-cdk-automation added the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Apr 27, 2024
Copy link
Contributor

mergify bot commented Apr 27, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation aws-cdk-automation removed the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Apr 27, 2024
@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: d307058
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify mergify bot merged commit 247aa35 into aws:main Apr 27, 2024
12 checks passed
Copy link
Contributor

mergify bot commented Apr 27, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation
Copy link
Collaborator

Comments on closed issues and PRs are hard for our team to see. If you need help, please open a new issue that references this one.

@aws aws locked as resolved and limited conversation to collaborators Jul 25, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
contribution/core This is a PR that came from AWS. effort/medium Medium work item – several days of effort p2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

(AwsCustomResource): fails to use latest SDK version
5 participants