Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

(events-target): Rule.addTarget(...) to add a CloudWatch Logs Group target to an EventBridge Rule is creating a Custom Lambda resource which is consistently timing out, leading to CFN deployment failures in China partition regions #29373

Closed
GavinZZ opened this issue Mar 6, 2024 · 1 comment · Fixed by #29374
Closed

(events-target): Rule.addTarget(...) to add a CloudWatch Logs Group target to an EventBridge Rule is creating a Custom Lambda resource which is consistently timing out, leading to CFN deployment failures in China partition regions #29373

GavinZZ opened this issue Mar 6, 2024 · 1 comment · Fixed by #29374
Assignees
@GavinZZ
Labels
@aws-cdk/aws-events-targets bug This issue is a bug. p1

Comments

@GavinZZ
Copy link
Contributor

GavinZZ commented Mar 6, 2024

Describe the bug

This logic is employed in approximately ~30 AWS regions, including commercial, china, and gov-cloud partition regions.

It looks like what the addTarget(...) action does is create a custom Lambda CFN resource (along with associated IAM policies) to perform the intended actions in our AWS accounts. However, as of at least ~Jan 11th, 2024, the custom Lambda resource has been consistently failing during our deployments in BJS and ZHY regions with the following error:

UPDATE_FAILED

CloudFormation did not receive a response from your Custom Resource. Please check your logs for requestId [9833c61e-99a2-4426-be43-189b3fa8613d]. If you are using the Python cfn-response module, you may need to update your Lambda function code so that CloudFormation can attach the updated version.

Diving deeper into the Lambda (AWS679f53fac002430cb0da5b7982bd22872D164C4C) logs for the custom resource shows the following logs:

START RequestId: 66de5d1d-40b2-49d2-b3a3-47761d6b9a98 Version: $LATEST

2024-02-27T16:15:34.662Z 66de5d1d-40b2-49d2-b3a3-47761d6b9a98 INFO Installing latest AWS SDK v2

2024-02-27T16:17:34.764Z 66de5d1d-40b2-49d2-b3a3-47761d6b9a98 Task timed out after 120.10 seconds

END RequestId: 66de5d1d-40b2-49d2-b3a3-47761d6b9a98

Expected Behavior

Should be able to deploy in CN regions

Current Behavior

Fails due to deployment time out, likely due to installing latest AWS SDK call time out in CN regions

Reproduction Steps

N/A

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.131.9

Framework Version

No response

Node.js Version

18

OS

MacOs

Language

TypeScript

Language Version

No response

Other information

No response
@GavinZZ GavinZZ added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Mar 6, 2024
@GavinZZ GavinZZ added needs-triage This issue or PR still needs to be triaged. and removed needs-triage This issue or PR still needs to be triaged. @aws-cdk/aws-events-targets labels Mar 6, 2024
@GavinZZ GavinZZ mentioned this issue Mar 6, 2024
Merged
1 task
@GavinZZ GavinZZ self-assigned this Mar 6, 2024
@GavinZZ GavinZZ added p1 @aws-cdk/aws-events-targets and removed needs-triage This issue or PR still needs to be triaged. labels Mar 6, 2024
@mergify mergify bot closed this as completed in #29374 Mar 6, 2024
mergify bot pushed a commit that referenced this issue Mar 6, 2024
@GavinZZ
fix(events_targets): installing latest aws sdk fails in cn partition (#…
f0383d6 
…29374)

### Issue # (if applicable)

Closes #29373

### Reason for this change

AWS Log Group event target by default installs the latest aws sdk for its custom resource and this would fail in `aws-cn` partition. This PR exposes the `installLatestAwsSdk` to the surface and allows users to optionally turn off `installLatestAwsSdk` for cloudwatch log events target.

### Description of changes

Allow users to override the value, if unset default to true which is the same behaviour as current.

### Description of how you validated changes

all tests pass.

### Checklist
- [x] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md)

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 6, 2024

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

@github-actions github-actions bot mentioned this issue Apr 1, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@GavinZZ GavinZZ

Labels
@aws-cdk/aws-events-targets bug This issue is a bug. p1
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(events_targets): installing latest aws sdk fails in cn partition aws/aws-cdk
1 participant
@GavinZZ