feat(cloudfront): s3 origin access control #31150

gracelu0 · 2024-08-19T16:35:47Z

Issue # (if applicable)

n/a

Reason for this change

Note: This PR is a WIP, not merging to the main branch. Merging to a feature branch first to facilitate smaller PR reviews.

Support OAC for S3 origins, see RFC for more details

Description of changes

Deprecate CloudFrontWebDistribution in favour of existing class Distribution
Add new abstract base class OriginAccessControlBase and subclass S3OriginAccessControl
Deprecate S3Origin in favour of new classes S3BucketOrigin and S3StaticWebsiteOrigin
Add new abstract class S3BucketOrigin
- 3 static functions for instantiating S3 origin with OAI, OAC and no origin access control (using bucket defaults)
Add new class S3StaticWebsiteOrigin
Update README for cloudfront and cloudfront-origins

Description of how you validated changes

integ.s3-origin-oac: basic use case setting up a distribution with an S3 origin with OAC, using assertions to check the distribution has the oac configured and the oac has the expected default properties
unit tests for S3OriginAccessControl

(more tests to follow in coming PRs)

Checklist

My code adheres to the CONTRIBUTING GUIDE and DESIGN GUIDELINES

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

…tOrigin

samson-keung · 2024-08-19T21:51:58Z

packages/aws-cdk-lib/aws-cloudfront-origins/README.md

+replace the `S3Origin` construct (now deprecated) with `S3BucketOrigin.withOriginAccessControl()` which automatically
+creates and sets up a OAC for you.
+The OAI will be deleted as part of the
+stack update. The logical IDs of the resources managed by


I am concerned that removing OAI permission and adding OAC permission in a single CFN stack deployment will create a short period of time where both permissions are not present for the bucket in rare situation. I have experienced a case where when IAM policy changed via CFN, even though the CFN reports update complete, the IAM policy still isn't truly reflected when invoking the system.

We should update this part of the README to recommend a 2-step migration approach. I will create a todo task to track this.

samson-keung · 2024-08-19T22:47:37Z

packages/aws-cdk-lib/aws-cloudfront/lib/origin-access-control.ts

+   * Imports an S3 origin access control from its id.
+   */
+  public static fromOriginAccessControlId(scope: Construct, id: string, originAccessControlId: string): IOriginAccessControl {
+    class Import extends Resource implements IOriginAccessControl {


I believe this should be class Import extends OriginAccessControlBase implements IOriginAccessControl

That is the case for Function:
https://github.com/gracelu0/aws-cdk/blob/b4377a5a3ad6d30730a084de6bc29c909a5c0e9d/packages/aws-cdk-lib/aws-lambda/lib/function.ts#L708

Yep makes sense, I'll update it

samson-keung · 2024-08-19T22:54:11Z

packages/aws-cdk-lib/aws-cloudfront/lib/origin.ts

+  /**
+   * The identifier of the Distribution this Origin is used for.
+   */
+  readonly distributionId: string;


This new attribute in this interface is a required attribute. This is a breaking change I believe because classes that implements this interface will need to be updated to have distributionId. Maybe can we make this optional?

Yes good catch, will update to be optional

samson-keung · 2024-08-19T23:02:34Z

packages/aws-cdk-lib/aws-cloudfront/lib/web-distribution.ts

@@ -1118,6 +1134,9 @@ export class CloudFrontWebDistribution extends cdk.Resource implements IDistribu

    let s3OriginConfig: CfnDistribution.S3OriginConfigProperty | undefined;
    if (originConfig.s3OriginSource) {
+      if (originConfig.s3OriginSource.originAccessIdentity && originConfig.s3OriginSource.originAccessControl) {
+        throw Error('Only one of origin access identity or origin access control can be defined.');
+      }


Would this block users from doing 2-step migration approach where they would have both OAI and OAC?

Removed this as we are deprecating CloudFrontWebDistribution - was an old change I accidentally pushed.

github-actions · 2024-08-19T23:30:41Z

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.

samson-keung · 2024-08-20T23:23:11Z

packages/aws-cdk-lib/aws-cloudfront-origins/lib/s3-bucket-origin.ts

+        // Failed to update bucket policy, assume using imported bucket
+        if (!result.statementAdded) {
+          Annotations.of(scope).addWarningV2('@aws-cdk/aws-cloudfront-origins:updateBucketPolicy',
+            'Cannot update bucket policy of an imported bucket. Set overrideImportedBucketPolicy to true or update the policy manually instead.');


I searched and cannot find info on overrideImportedBucketPolicy. Was the plan to have it as a feature flag?

Imo, we should just throw the warning as feature flag adds to the code complexity.

Ah yes it was part of the previous design, I'll remove since we are not updating policies of imported buckets.

samson-keung · 2024-08-20T23:28:43Z

packages/aws-cdk-lib/aws-cloudfront-origins/lib/s3-bucket-origin.ts

+          // TODO: update this warning
+          Annotations.of(scope).addWarningV2('@aws-cdk/aws-cloudfront-origins:updateKeyPolicy',
+            'If you run into a circular dependency issue during deployment, please refer to README for instructions.');
+        }


This warning probably will leave users scratching their head because when they see this warning, they wouldn't have seen the circular dependency error.

We should update this to change the key policy. It will allow users to have other workarounds.

Do you mean add to the key policy here and let them run into the circular dependency issue? Then at that time they would see the warning message and follow it to configure a workaround

samson-keung · 2024-08-22T18:35:37Z

packages/aws-cdk-lib/aws-cloudfront-origins/lib/s3-bucket-origin.ts

+  * An optional Origin Access Control
+  * @default - an Origin Access Control will be created.
+  */
+  readonly originAccessControl?: cloudfront.IOriginAccessControl;


I found that this interface follows this "extends path":
S3BucketOriginWithOACProps -> S3BucketOriginBaseProps -> cloudfront.OriginProps -> OriginOptions

And in OriginOptions, there is a prop originAccessControlId. So the user can do:

origins.S3BucketOrigin.withOriginAccessControl(bucket, { originAccessControlId: 'oacId123', originAccessControl: new cloudfront.S3OriginAccessControl(stack, 'MyOac'), })

Then in the CFN template, the originAccessControlId will be ignored.

Good catch, we should remove originAccessControlId from OriginOptions and only allow user to configure OAC with object of type IOriginAccessControl. This follows the design guidelines to use strong typing when referencing resources: https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md#construct-interface.

gracelu0 and others added 22 commits August 19, 2024 09:13

wip oac

5a2a62f

create custom resource to update kms policy

a3e0a5d

custom resource for bucket policy

ae5790a

Support oac in webDistribution

ff3c29c

refactor

901ecc1

fix undefined distribution id

b8a926d

refactor

8e20799

Add validation for origin type on OAC

2f13ef3

Add origin type to oac

f5690a8

remove feature flag

b5706c9

deprecate S3Origin and replace with S3StaticWebsiteOrigin and S3Bucke…

3697c6b

…tOrigin

Add S3BucketOrigin subclass props

df6667d

update README for cloudfront with new API for S3BucketOrigin

37a4b59

add unit tests for S3 OAC

9daa59b

remove name length validation (leave this to cloudformation)

032a032

fix unit test

29ae20c

formatting

0ac34fb

Add integration test for S3 bucket origin with default OAC

1020955

add if check to OAC bind method

63097d7

check default properties in OriginAccessControl in unit test

12cb4ad

remove custom resource

09dc52e

fix integ test formatting

3cd0605

aws-cdk-automation requested a review from a team August 19, 2024 16:36

github-actions bot added the p2 label Aug 19, 2024

mergify bot added the contribution/core This is a PR that came from AWS. label Aug 19, 2024

gracelu0 added 2 commits August 19, 2024 12:17

Add note about bucket object ownership with OAC

9cacc58

remove web distribution oac

0acc247

samson-keung requested changes Aug 19, 2024

View reviewed changes

gracelu0 merged commit 7e2dea7 into aws:gracelu0/s3-oac-l2 Aug 19, 2024
7 checks passed

github-actions bot locked as resolved and limited conversation to collaborators Aug 19, 2024

samson-keung reviewed Aug 20, 2024

View reviewed changes

samson-keung reviewed Aug 22, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(cloudfront): s3 origin access control #31150

feat(cloudfront): s3 origin access control #31150

gracelu0 commented Aug 19, 2024 •

edited

Loading

samson-keung Aug 19, 2024

samson-keung Aug 19, 2024

gracelu0 Aug 19, 2024

samson-keung Aug 19, 2024

gracelu0 Aug 21, 2024

samson-keung Aug 19, 2024

gracelu0 Aug 19, 2024

github-actions bot commented Aug 19, 2024

samson-keung Aug 20, 2024

gracelu0 Aug 20, 2024

samson-keung Aug 20, 2024

gracelu0 Aug 21, 2024

samson-keung Aug 22, 2024

gracelu0 Aug 22, 2024

feat(cloudfront): s3 origin access control #31150

feat(cloudfront): s3 origin access control #31150

Conversation

gracelu0 commented Aug 19, 2024 • edited Loading

Issue # (if applicable)

Reason for this change

Description of changes

Description of how you validated changes

Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Aug 19, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gracelu0 commented Aug 19, 2024 •

edited

Loading