(aws-cloudfront): Bucket policy permissions broke CloudFront integration

[ferdingler]

After upgrading @aws-cdk/aws-cloudfront from v1.74.0 to v1.90.1 our CloudFront distribution was no longer able to access contents on the S3 bucket via OAI getting an AccessDenied error.

After doing some research I realized that the bucket policy changed from this:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity E11N7NL3KG1PGX"
            },
            "Action": [
                "s3:GetObject*",
                "s3:GetBucket*",
                "s3:List*"
            ],
            "Resource": [
                "arn:aws:s3:::my-bucket",
                "arn:aws:s3:::my-bucket/*"
            ]
        }
    ]
}

to this:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity E11N7NL3KG1PGX"
            },
            "Action": [
                "s3:GetObject*"
            ],
            "Resource": [
                "arn:aws:s3:::my-bucket/*"
            ]
        }
    ]
}

Accessing the CloudFront URL from the root works fine https://123.cloudfront.net but the problem is when accessing a specific path like https://123.cloudfront.net/admin then it gives an AccessDenied error.

Reproduction Steps

Upgraded @aws-cdk/aws-cloudfront to v1.90.1

What did you expect to happen?

CloudFront distribution should be able to continue to access objects in the S3 bucket.

What actually happened?

Accessing CloudFront url now gives an Access Denied error:

<Error>
<Code>AccessDenied</Code>
<Message>Access Denied</Message>
<RequestId>YN2354KR1VQK4MM0</RequestId>
<HostId>sZTCl9TPf37VT6cR4ytu2kkd/86tZjTD9IOJx/2av/apAUxQm4KO04iXXd99dakJZre0FWLF9s0=</HostId>
</Error>

Environment

• Framework Version: v1.90.1

---

This is 🐛 Bug Report

[ferdingler]

I found this PR is where the new bucket policy was introduced: #13087 . Tagging @njlynch and @nija-at for visibility.

[ferdingler]

As described by @njlynch in the PR, the official documentation states that only s3:GetObject permission is required when configuring CloudFront and S3 via Origin Access Identity. However, our CloudFront distribution definitely broke with this change. If I manually add back the s3:List* permission to the bucket policy, it works fine.

[robertd]

Hmm interesting... I've pushed this same change to all of our CF distros this morning w/o any issues (went from cdk 1.89.0 to 1.91.0 though). Just out of curiosity can you please check if your CloudFormation stack drifted by any chance? Also, have you also created a bucket using the same stack... or is it an imported pre-existing standalone bucket?

...
[~] AWS::S3::BucketPolicy dynamap-test-bucket/Policy dynamaptestbucketPolicy974ACB96
 └─ [~] PolicyDocument
     └─ [~] .Statement:
         └─ @@ -25,11 +25,7 @@
            [ ]   "Sid": "HttpsOnly"
            [ ] },
            [ ] {
            [-]   "Action": [
            [-]     "s3:GetObject*",
            [-]     "s3:GetBucket*",
            [-]     "s3:List*"
            [-]   ],
            [+]   "Action": "s3:GetObject",
            [ ]   "Effect": "Allow",
            [ ]   "Principal": {
            [ ]     "CanonicalUser": {
            @@ -39,27 +35,19 @@
            [ ]         ]
            [ ]       }
            [ ]     },
            [-]     "Resource": [
            [-]       {
            [-]         "Fn::GetAtt": [
            [-]           "dynamaptestbucket7FE81451",
            [-]           "Arn"
            [+]     "Resource": {
            [+]       "Fn::Join": [
            [+]         "",
            [+]         [
            [+]           {
            [+]             "Fn::GetAtt": [
            [+]               "dynamaptestbucket7FE81451",
            [+]               "Arn"
            [+]             ]
            [+]           },
            [+]           "/*"
            [ ]         ]
            [-]       },
            [-]       {
            [-]         "Fn::Join": [
            [-]           "",
            [-]           [
            [-]             {
            [-]               "Fn::GetAtt": [
            [-]                 "dynamaptestbucket7FE81451",
            [-]                 "Arn"
            [-]               ]
            [-]             },
            [-]             "/*"
            [-]           ]
            [-]         ]
            [-]       }
            [-]     ]
            [+]       ]
            [+]     }
            [ ]   }
            [ ] ]

[njlynch]

@ferdingler - Sorry for the breakage here!

While we work on finding the right balance and fix here, the work-around mentioned in the linked PR should work to get you back up and running:

// Before
const bucket = new s3.Bucket(this, 'Bucket');
const s3Origin = new origins.S3Origin(this, 'S3Origin', { bucket });

// After
const bucket = new s3.Bucket(this, 'Bucket');
const originAccessIdentity = new cloudfront.OriginAccessIdentity(this, 'OAI');
bucket.grantRead(originAccessIdentity);
const s3Origin = new origins.S3Origin(this, 'S3Origin', { bucket, originAccessIdentity });

I'd like to learn more about your use case, as I'm not sure I understand what the behavior was like for you prior to the upgrade.

Accessing the CloudFront URL from the root works fine https://123.cloudfront.net but the problem is when accessing a specific path like https://123.cloudfront.net/admin then it gives an AccessDenied error.

When you say https://123.cloudfront.net/admin, do you mean like that literally (just the directory) or anything in that subfolder (e.g., https://123.cloudfront.net/admin/secretadminstuff.html) as well? Based on my understanding -- and AWS blog posts like this -- having a default root object (index.html) only works for the root of the distribution, and not subdirectories; this means accessing /admin/ doesn't redirect to /admin/index.html. In my tests, no matter what bucket policy I set, accessing a subfolder like /admin/ yields an empty 200 response.

% curl -i http://d1234567890ab.cloudfront.net/admin/
HTTP/1.1 200 OK
Content-Type: application/x-directory; charset=UTF-8
Content-Length: 0
...

Is this all under a single behavior and origin, or are there multiple behaviors/origins set up? If so, is the behavior for /admin different from the one for the root ('/')?

The only impact I can see from the bucket policy change is when visiting the domain root without a default root object; before, you got a XML listing of the contents, and after, you get an access denied message. I'd love to better understand what else changes that I haven't yet been able to discover or reproduce.

[ferdingler]

@njlynch thanks for the recommended work-around. Here is some additional information about our usecase:

We have a ReactJS application hosted on the S3 bucket that is fronted by the CloudFront distribution via OAI. The S3 bucket has encryption enabled using the default S3 master-key SSE-S3. The CloudFront distribution has only 1 origin (the S3 bucket) and it has index.html configured as default root object.

Additionally, there is a Lambda@Edge function that runs on the CloudFront distribution on Origin Response that appends some security headers to the CloudFront responses.

Our codebase is open source, so you can take a look at the CDK construct that creates the setup I just described if it helps:
https://github.com/awslabs/performance-dashboard-on-aws/blob/cb65206b819ea043742f396f2c4fbe664ebc5257/cdk/lib/frontend-stack.ts#L58

When you say https://123.cloudfront.net/admin, do you mean like that literally (just the directory) or anything in that subfolder (e.g., https://123.cloudfront.net/admin/secretadminstuff.html) as well?

I'm puzzled by this as well because like I mentioned above, we have a Single Page Application (React) that once it loads on the user's browser, any path after the domain name should be handled by react router within the app itself and it shouldn't trigger another request to CloudFront. I need to dive deeper on this part because I can't explain why it works on the root domain but it doesn't work on any path after that.

What is true though is that adding back the s3:List* permission and the resource arn:aws:s3:::my-bucket to the bucket policy makes it work again.

[njlynch]

Ah, the code is super helpful, thanks!

In your setup before, a user going to /admin/ triggers a 404. You have a custom error handler for on 404s which redirects to index.html (the SPA root). So visiting /admin -> 404 -> index.html -> React renders admin screen. With the bucket policy change, trying to navigate to /admin/ results in a 403 instead of 404. The error handler isn't configured for 403s, so you get the "native" CloudFormation Access Denied message. If a 403 error handler is set up, everything will work as expected.

[njlynch]

Per offline conversation, I'm going to close this out. I think the above error handler change is a reasonable workaround. Please feel free to reopen if you have further questions.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[ferdingler]

Thanks for the quick response on this ticket @njlynch . Just in case anyone runs into this situation in the future, here is the change we have to make in our CloudFront distribution as per Nick's suggestion:

const distribution = new cloudFront.CloudFrontWebDistribution(
      this,
      "CloudFrontDistribution",
      {
        errorConfigurations: [
          {
            errorCode: 404,
            responseCode: 200,
            responsePagePath: "/index.html",
          },
          {
            errorCode: 403, // this is the new addition due to the bucket policy returning 403
            responseCode: 200,
            responsePagePath: "/index.html",
          },
        ],
        originConfigs: [
          {
            s3OriginSource: {
              s3BucketSource: this.frontendBucket,
              originAccessIdentity: originAccess,
            },
          },
        ],
      }
    );