Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

aws-s3: Expose SkipDestinationValidation option when setting bucket notifications #30914

Closed
1 of 2 tasks
yerzhan7 opened this issue Jul 22, 2024 · 3 comments · Fixed by #30916 or mannjaro/serverless-bedrock-proxy#2 · May be fixed by NOUIY/aws-solutions-constructs#120, NOUIY/aws-solutions-constructs#121 or NOUIY/aws-solutions-constructs#128
Closed
1 of 2 tasks

aws-s3: Expose SkipDestinationValidation option when setting bucket notifications #30914

yerzhan7 opened this issue Jul 22, 2024 · 3 comments · Fixed by #30916 or mannjaro/serverless-bedrock-proxy#2 · May be fixed by NOUIY/aws-solutions-constructs#120, NOUIY/aws-solutions-constructs#121 or NOUIY/aws-solutions-constructs#128
Labels
@aws-cdk/aws-s3 Related to Amazon S3 effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p2

Comments

@yerzhan7
Copy link
Contributor

yerzhan7 commented Jul 22, 2024 •
edited
Loading

Describe the feature

S3 API: PutBucketNotificationConfiguration

https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketNotificationConfiguration.html

When customers call this API to setup S3 notification configuration for SQS/SNS/Lambda S3 sends s3:TestEvent in order to validate permissions. (For Lambda it does dryrun function invocation instead)

However, some customers do not want to do that and test permissions during CDK deployment.

  1. Internal reference: 49359101-0e5e-43f3-99eb-3c6c5ed68db1

Use Case

One customer does not want these test events because they have alarm on unconsumed messages in SQS and they do not have any SQS consumers. And they update notification configuration frequently, which leads to many test events in the queue.

See internal ticket: P142186522

Proposed Solution

Add new Bucket property with this flag.

Other Information

No response

Acknowledgements

  • I may be able to implement this feature request
  • This feature might incur a breaking change

CDK version used

latest

Environment details (OS name and version, etc.)

MacOS
@yerzhan7 yerzhan7 added feature-request A feature should be added or improved. needs-triage This issue or PR still needs to be triaged. labels Jul 22, 2024
@github-actions github-actions bot added the @aws-cdk/aws-s3 Related to Amazon S3 label Jul 22, 2024
yerzhan7 added a commit to yerzhan7/aws-cdk that referenced this issue Jul 22, 2024
@yerzhan7
feat(s3): add skip destination validation property
89a5d9b 
Closes aws#30914
@yerzhan7 yerzhan7 mentioned this issue Jul 22, 2024
Merged
1 task
@pahud
Copy link
Contributor

pahud commented Jul 22, 2024

Thank you for your pull request!

@pahud pahud added p2 effort/medium Medium work item – several days of effort and removed needs-triage This issue or PR still needs to be triaged. labels Jul 22, 2024
This was referenced Jul 29, 2024
Closed
Closed
@mergify mergify bot closed this as completed in #30916 Aug 19, 2024
@mergify mergify bot closed this as completed in ffc40e5 Aug 19, 2024
@github-actions GitHub Actions
Copy link

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.

1 similar comment
@github-actions GitHub Actions
Copy link

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 19, 2024
GavinZZ pushed a commit that referenced this issue Aug 20, 2024
@yerzhan7 @GavinZZ
feat(s3): add skip destination validation property (#30916)
efd1da4 
Closes #30914.

When customers call this API to setup S3 notification configuration for SQS/SNS/Lambda S3 sends s3:TestEvent in order to validate permissions. (For Lambda it does dryrun function invocation instead)

However, some customers do not want to do that and test permissions during CDK deployment.

Internal reference: `49359101-0e5e-43f3-99eb-3c6c5ed68db1`

For example, one customer does not want these test events because they have alarm on unconsumed messages in SQS and they do not have any SQS consumers. And they update notification configuration frequently, which leads to many test events in the queue. See internal ticket: `P142186522`

Expose skip destination validation property when calling PutBucketNotification API in Bucket props.

Unit test updated.

Updated integration tests. Note that 2 integration tests I had to fix and run them with `--disable-update-workflow` flag because they were failing:
 - `integ.s3.imported-bucket.js` test failed because someone already created bucket `cdk-integration-test-s3-imported-bucket-name`
 - `integ.bucket-notifications.js` test failed because of overlapping suffix error (not sure how it was passing previously):

 ```
❌  cdk-integ-lambda-bucket-s3-notifications failed: Error: The stack named cdk-integ-lambda-bucket-s3-notifications failed to deploy: UPDATE_FAILED
(The following resource(s) failed to update: [Construct1IntegUnmanagedBucket1Notifications4A1599D7]. ): Received response status [FAILED] from custom resource.
Message returned: Error: An error occurred (InvalidArgument) when calling the PutBucketNotificationConfiguration operation: Configuration is ambiguously defined.
Cannot have overlapping suffixes in two rules if the prefixes are overlapping for the same event type..
See the details in CloudWatch Log Stream: 2024/07/22/[$LATEST]e6a16cf979dd4671998e7d911769ff42 (RequestId: 19f6fcd7-d31d-4fbf-9f4a-e3b7cba1cd2b),
Rolling back the failed resource only., Received response status [FAILED] from custom resource. Message returned: Error: An error occurred
(InvalidArgument) when calling the PutBucketNotificationConfiguration operation: Configuration is ambiguously defined.
Cannot have overlapping suffixes in two rules if the prefixes are overlapping for the same event type..
See the details in CloudWatch Log Stream: 2024/07/22/[$LATEST]c30efd0375d64b8088e0ee64d63ce4db (RequestId: 19f6fcd7-d31d-4fbf-9f4a-e3b7cba1cd2b)
```

- [X] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md)

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
@aws-cdk/aws-s3 Related to Amazon S3 effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p2
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat(s3): add skip destination validation property yerzhan7/aws-cdk
[Snyk] Upgrade aws-cdk-lib from 2.151.0 to 2.154.1 mannjaro/serverless-bedrock-proxy
[Snyk] Upgrade @aws-cdk/cloudformation-diff from 2.99.1 to 2.155.0 NOUIY/aws-solutions-constructs
[Snyk] Upgrade aws-cdk from 2.99.1 to 2.155.0 NOUIY/aws-solutions-constructs
[Snyk] Upgrade @aws-cdk/cloudformation-diff from 2.99.1 to 2.156.0 NOUIY/aws-solutions-constructs
[Snyk] Upgrade aws-cdk from 2.99.1 to 2.156.0 NOUIY/aws-solutions-constructs
[Snyk] Upgrade @aws-cdk/cloudformation-diff from 2.99.1 to 2.157.0 NOUIY/aws-solutions-constructs
[Snyk] Upgrade aws-cdk from 2.99.1 to 2.157.0 NOUIY/aws-solutions-constructs
[Snyk] Upgrade @aws-cdk/cloudformation-diff from 2.99.1 to 2.158.0 NOUIY/aws-solutions-constructs
[Snyk] Upgrade aws-cdk from 2.99.1 to 2.158.0 NOUIY/aws-solutions-constructs
2 participants
@pahud @yerzhan7