fix(s3): unable to update the s3 event notifications on an existing S3 bucket

[xazhao]

Issue # (if applicable)

Closes #31303.

Reason for this change

During the update S3 bucket notifications, if there are multiple filter rules involved, the order of filter rules is different from CFN event notifications and S3 get notifications call. This will result different hash value which makes CDK unable to recognize s3 notifications created by the stack itself.

Description of changes

Sort the notifications filter rules by key when calculating the hash value.

Description of how you validated changes

Manually tested and integration tests.

Checklist

• My code adheres to the CONTRIBUTING GUIDE and DESIGN GUIDELINES

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

[aws-cdk-automation]

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.

[xenonlcs]

Hello everyone, here are my 2 cents. THIS IS NOT RELATED TO SORT PROBLEMS (or not only)

In CDK v2.3 the method used to verify if a notification configuration was external was based on the following check (this is from the Custom Resource behind aws_s3_notifications.LambdaDestination resource):
external_notifications[t] = [n for n in existing_notifications.get(t, []) if not n['Id'].startswith(f"{stack_id}-")]

In CDK v2.136 the method is this one (not checked when this change was introduced):

  ids = [with_id(n) for n in old.get(t, [])]
  old_incoming_ids = [n['Id'] for n in ids]
  external_notifications[t] = [n for n in existing_notifications.get(t, []) if not n['Id'] in old_incoming_ids]

The function with_id(n) is this one:

def with_id(n):
    n['Id'] = f"{stack_id}-{hash(json.dumps(n, sort_keys=True))}"
    return n

The built-in function hash does not return the same value across different instances of python even if the input is the same.

The Custom Resource will treat as external_notifications even the ones that are not, leading to duplicates

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

[xazhao]

Hello everyone, here are my 2 cents. THIS IS NOT RELATED TO SORT PROBLEMS (or not only)

In CDK v2.3 the method used to verify if a notification configuration was external was based on the following check (this is from the Custom Resource behind aws_s3_notifications.LambdaDestination resource): external_notifications[t] = [n for n in existing_notifications.get(t, []) if not n['Id'].startswith(f"{stack_id}-")]

In CDK v2.136 the method is this one (not checked when this change was introduced):

  ids = [with_id(n) for n in old.get(t, [])]
  old_incoming_ids = [n['Id'] for n in ids]
  external_notifications[t] = [n for n in existing_notifications.get(t, []) if not n['Id'] in old_incoming_ids]

The function with_id(n) is this one:

def with_id(n):
    n['Id'] = f"{stack_id}-{hash(json.dumps(n, sort_keys=True))}"
    return n

The built-in function hash does not return the same value across different instances of python even if the input is the same.

The Custom Resource will treat as external_notifications even the ones that are not, leading to duplicates

This comment has been addressed in the issue discussion: #31303 (comment)

[GavinZZ]

I think the changes LGTM, thanks for fixing the issue.

[GavinZZ]

I approved the PR but added a do-not-merge as I think @shikha372 is also reviewing the PR. Would wait for her reviews.

[shikha372]

Thanks @GavinZZ , discussed with @xazhao , we would like to confirm one that its not affecting anything in custom resources that are related to this and also test out a successful update scenario before merging.
Also, from the analysis, i really think we should check there will be no impact of this sorting to the external customers or do anything that causes replacement of existing filter rules, since I see some change in SQS related snapshots as well.

Here's an example to illustrate a potential regression (from analysis):

const bucket = new s3.Bucket(this, 'MyBucket');

// Existing notification configuration
bucket.addEventNotification(
  s3.EventType.OBJECT_CREATED,
  new s3n.LambdaDestination(myLambda),
  { prefix: 'uploads/', suffix: '.jpg' }
);

// After deploying with the old logic, the notification has a specific hash identifier

// When updating the stack with the new logic:
// The same notification might be perceived as different due to the new sorting method
// CDK might attempt to remove the "old" notification and create a "new" one
// This could lead to temporary loss of notifications or duplicate notifications

[mergify]

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

[aws-cdk-automation]

AWS CodeBuild CI Report

• CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
• Commit ID: e4e14aa
• Result: SUCCEEDED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[mergify]

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

[github-actions]

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.