[s3 + cloudfront] Allow cross-region references

[justin8]

This sort of applies to both S3 and CloudFront; I was attempting to set up CloudFront distribution with origins in 2 different regions. When I refer to the bucket it is using the incorrect regional endpoint by assuming the bucket access endpoint should be the region of the stack/distribution and not the region of the bucket.

Reproduction Steps

    // This stack is deployed in us-east-1 so that we can generate ACM certs and actually consume them in cloudfront

    const sydBucket = s3.Bucket.fromBucketName(this, "sydBucket", "bucket-from-ap-southeast-2")
    const pdxBucket = s3.Bucket.fromBucketName(this, "pdxBucket", "bucket-from-us-west-2")

    const dist = new cloudfront.Distribution(this, "dist", {
        defaultBehavior: {
            origin: new origins.OriginGroup({
                primaryOrigin: new origins.S3Origin(sydBucket),
                fallbackOrigin: new origins.S3Origin(pdxBucket),
            });
        });

What did you expect to happen?

When I reference a bucket in the region that the stack is not in, and use it as a CloudFront origin I expect it to use the correct regional endpoint and not an invalid one. e.g. using ${bucketName}.s3-${region}.amazonaws.com

What actually happened?

This results in the origin group being created as expected, but the 2 S3 origins use the following path:

bucket-from-ap-southeast-2.s3.us-east-1.amazonaws.com
bucket-from-us-west-2.s3.us-east-1.amazonaws.com

Obviously this does not work, and results in an error when accessing the distribution:

<Error>
<Code>PermanentRedirect</Code>
<Message>
The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.
</Message>
<Endpoint>
bucket-from-ap-southeast-2.s3-ap-southeast-2.amazonaws.com
</Endpoint>
<Bucket>bucket-from-ap-southeast-2</Bucket>
<RequestId>96108733F5AA5E03</RequestId>
<HostId>
FI6ZaXxn7QLDO4lV6l7AbIlXKgSJk72ZiIj78AFJgREUCel3R6Kk9hu9tZhNOoa/OWSNi7Ta6x0=
</HostId>
</Error>

Environment

• CLI Version: 1.57
• Framework Version: 1.57
• Node.js Version: v12.13.1
• OS: MacOS
• Language (Version): all

Other

---

This is 🐛 Bug Report

[njlynch]

As a general rule, resources within a stack are considered to be within the same region. You'll note that Bucket.fromBucketAttributes retrieves the region from the current stack:

aws-cdk/packages/@aws-cdk/aws-s3/lib/bucket.ts

Lines 1134 to 1137 in 7432ab0

	public static fromBucketAttributes(scope: Construct, id: string, attrs: BucketAttributes): IBucket {
	const stack = Stack.of(scope);
	const region = stack.region;
	const urlSuffix = stack.urlSuffix;

One option as a workaround is to explicitly supply the bucketRegionalDomainName when importing the bucket:

const sydBucket = s3.Bucket.fromBucketAttributes(this, "sydBucket", {
  bucketName: "bucket-from-ap-southeast-2",
  bucketRegionalDomainName: "bucket-from-ap-southeast-2.s3.ap-southeast-2.amazonaws.com",
});

However, as long as you're doing that -- and don't need the imported bucket for other means -- you could just skip the bucket altogether and use an HttpOrigin:

const dist = new cloudfront.Distribution(this, "dist", {
    defaultBehavior: {
        origin: new origins.OriginGroup({
            primaryOrigin: new origins.HttpOrigin("bucket-from-ap-southeast-2.s3.ap-southeast-2.amazonaws.com", {
                protocolPolicy: cloudfront.OriginProtocolPolicy.HTTP_ONLY,
            }),
            fallbackOrigin: new origins.HttpOrigin("bucket-from-us-west-2.s3.us-west-2.amazonaws.com", {
                protocolPolicy: cloudfront.OriginProtocolPolicy.HTTP_ONLY,
            }),
        });
    });

I'm going to go ahead and close this, as it's not a bug, but feel free to re-open if you have further questions on the above.

[justin8]

That's kind of my point with this issue.

CloudFront is an inherently global service; although you must create it in us-east-1 if you want a custom certificate.

The CDK interface for managing global resources such as this still follows a lot of the limitations of CloudFormation.

What I'd love to see is something like what we have for dynamo global tables where we can actually use them, unlike in raw CloudFormation.

Perhaps this is as simple as allowing a virtual reference to a bucket in another region; you can't do cross-region stack exports still, and a common technique to avoiding the need of them is to use xx.fromArn or similar functions to refer to an object where it's not created in the current app or even the CDK at all; but there is no way to reference an S3 source other than as you showed; using a Http origin and making customers figure out the correct regional endpoints and such themselves and then hardcoding them.

Also that example would not work with an access identity since it would be a custom origin not an S3 origin.

edit: I can't re-open the issue, it seems that has been disabled; but I still believe this is a valid issue that should be open. My only option is to raise a new issue and reference this one it seems.

[justin8]

Should I just make a new issue? It would be cleaner to just re-open this, but I have no permissions to do so.

[njlynch]

Re-opening (apologies -- I thought requesters could re-open closed issues):

Thanks for pushing back a bit here. After taking a deeper look, I think that allowing you to specify the region of an imported bucket is a reasonable request, and fairly small at that. The proposal would be to allow either of the below to work:

const myImportedBucket = Bucket.fromArn(this, 'Bucket1', 'aws:s3:us-east-1:...'); // Sets region from ARN
const myOtherImportedBucket = Bucket.fromBucketAttributes(this, 'Bucket2', {
  bucketName: 'myBucket1235',
  region: 'us-east-1', // Set region explicitly
}); 

I'll post a PR for this shortly and confer with the team on the above.

[justin8]

Awesome! Appreciate the fast feedback as always.