Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(s3-deployment): control object access #15730

Merged
merged 5 commits into from
Aug 2, 2021
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,8 @@
"@aws-cdk/aws-eks/yaml/**",
"@aws-cdk/aws-events-targets/aws-sdk",
"@aws-cdk/aws-events-targets/aws-sdk/**",
"@aws-cdk/aws-s3-deployment/case",
"@aws-cdk/aws-s3-deployment/case/**",
"@aws-cdk/cloud-assembly-schema/jsonschema",
"@aws-cdk/cloud-assembly-schema/jsonschema/**",
"@aws-cdk/cloud-assembly-schema/semver",
Expand Down
30 changes: 30 additions & 0 deletions packages/@aws-cdk/aws-s3-deployment/NOTICE
Original file line number Diff line number Diff line change
@@ -1,2 +1,32 @@
AWS Cloud Development Kit (AWS CDK)
Copyright 2018-2021 Amazon.com, Inc. or its affiliates. All Rights Reserved.

-------------------------------------------------------------------------------

The AWS CDK includes the following third-party software/licensing:

** case - https://www.npmjs.com/package/case
Copyright (c) 2013 Nathan Bubna

Permission is hereby granted, free of charge, to any person
obtaining a copy of this software and associated documentation
files (the "Software"), to deal in the Software without
restriction, including without limitation the rights to use,
copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following
conditions:

The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.

----------------
30 changes: 18 additions & 12 deletions packages/@aws-cdk/aws-s3-deployment/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,17 +147,22 @@ User-defined metadata are not used by S3 and keys always begin with `x-amz-meta-

System defined metadata keys include the following:

- cache-control
- content-disposition
- content-encoding
- content-language
- content-type
- expires
- server-side-encryption
- storage-class
- website-redirect-location
- ssekms-key-id
- sse-customer-algorithm
- cache-control (`--cache-control` in `aws s3 sync`)
- content-disposition (`--content-disposition` in `aws s3 sync`)
- content-encoding (`--content-encoding` in `aws s3 sync`)
- content-language (`--content-language` in `aws s3 sync`)
- content-type (`--content-type` in `aws s3 sync`)
- expires (`--expires` in `aws s3 sync`)
- x-amz-storage-class (`--storage-class` in `aws s3 sync`)
- x-amz-website-redirect-location (`--website-redirect` in `aws s3 sync`)
- x-amz-server-side-encryption (`--sse` in `aws s3 sync`)
- x-amz-server-side-encryption-aws-kms-key-id (`--sse-kms-key-id` in `aws s3 sync`)
- x-amz-server-side-encryption-customer-algorithm (`--sse-c-copy-source` in `aws s3 sync`)
- x-amz-acl (`--acl` in `aws s3 sync`)

You can find more information about system defined metadata keys in
[S3 PutObject documentation](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html)
and [`aws s3 sync` documentation](https://docs.aws.amazon.com/cli/latest/reference/s3/sync.html).

```ts
const websiteBucket = new s3.Bucket(this, 'WebsiteBucket', {
Expand All @@ -177,6 +182,7 @@ new s3deploy.BucketDeployment(this, 'DeployWebsite', {
storageClass: StorageClass.INTELLIGENT_TIERING,
serverSideEncryption: ServerSideEncryption.AES_256,
cacheControl: [CacheControl.setPublic(), CacheControl.maxAge(cdk.Duration.hours(1))],
accessControl: s3.BucketAccessControl.BUCKET_OWNER_FULL_CONTROL,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We usually like to add a couple of sentences on what this feature, when it should be used and any caveats.

Some require a separate section but most just a couple of sentences.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So accessControl is one of system metadata properties like serverSideEncryption and etc. It seems like there is no additional information provided for other system metadata properties here. I have added a note about how system metadata maps to aws s3 sync arguments and where additional information can be obtained.

});
```

Expand Down Expand Up @@ -230,7 +236,7 @@ size of the AWS Lambda resource handler.
## Development

The custom resource is implemented in Python 3.6 in order to be able to leverage
the AWS CLI for "aws sync". The code is under [`lib/lambda`](https://github.com/aws/aws-cdk/tree/master/packages/%40aws-cdk/aws-s3-deployment/lib/lambda) and
the AWS CLI for "aws s3 sync". The code is under [`lib/lambda`](https://github.com/aws/aws-cdk/tree/master/packages/%40aws-cdk/aws-s3-deployment/lib/lambda) and
unit tests are under [`test/lambda`](https://github.com/aws/aws-cdk/tree/master/packages/%40aws-cdk/aws-s3-deployment/test/lambda).

This package requires Python 3.6 during build time in order to create the custom
Expand Down
8 changes: 8 additions & 0 deletions packages/@aws-cdk/aws-s3-deployment/lib/bucket-deployment.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import * as lambda from '@aws-cdk/aws-lambda';
import * as s3 from '@aws-cdk/aws-s3';
import * as cdk from '@aws-cdk/core';
import { AwsCliLayer } from '@aws-cdk/lambda-layer-awscli';
import { kebab as toKebabCase } from 'case';
import { Construct } from 'constructs';
import { ISource, SourceConfig } from './source';

Expand Down Expand Up @@ -164,6 +165,12 @@ export interface BucketDeploymentProps {
* @see https://docs.aws.amazon.com/AmazonS3/latest/dev/ServerSideEncryptionCustomerKeys.html#sse-c-how-to-programmatically-intro
*/
readonly serverSideEncryptionCustomerAlgorithm?: string;
/**
* System-defined x-amz-acl metadata to be set on all objects in the deployment.
* @default - Not set.
* @see https://docs.aws.amazon.com/AmazonS3/latest/userguide/acl-overview.html#canned-acl
*/
readonly accessControl?: s3.BucketAccessControl;

/**
* The VPC network to place the deployment lambda handler in.
Expand Down Expand Up @@ -282,6 +289,7 @@ function mapSystemMetadata(metadata: BucketDeploymentProps) {
if (metadata.websiteRedirectLocation) { res['website-redirect'] = metadata.websiteRedirectLocation; }
if (metadata.serverSideEncryptionAwsKmsKeyId) { res['sse-kms-key-id'] = metadata.serverSideEncryptionAwsKmsKeyId; }
if (metadata.serverSideEncryptionCustomerAlgorithm) { res['sse-c-copy-source'] = metadata.serverSideEncryptionCustomerAlgorithm; }
if (metadata.accessControl) { res.acl = toKebabCase(metadata.accessControl.toString()); }

return Object.keys(res).length === 0 ? undefined : res;
}
Expand Down
4 changes: 4 additions & 0 deletions packages/@aws-cdk/aws-s3-deployment/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,7 @@
"@aws-cdk/aws-s3-assets": "0.0.0",
"@aws-cdk/lambda-layer-awscli": "0.0.0",
"@aws-cdk/core": "0.0.0",
"case": "1.6.3",
"constructs": "^3.3.69"
},
"homepage": "https://github.com/aws/aws-cdk",
Expand All @@ -110,6 +111,9 @@
"@aws-cdk/core": "0.0.0",
"constructs": "^3.3.69"
},
"bundledDependencies": [
"case"
],
"engines": {
"node": ">= 10.13.0 <13 || >=13.7.0"
},
Expand Down
38 changes: 38 additions & 0 deletions packages/@aws-cdk/aws-s3-deployment/test/bucket-deployment.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -325,6 +325,7 @@ test('system metadata is correctly transformed', () => {
websiteRedirectLocation: 'example',
cacheControl: [s3deploy.CacheControl.setPublic(), s3deploy.CacheControl.maxAge(cdk.Duration.hours(1))],
expires: expiration,
accessControl: s3.BucketAccessControl.BUCKET_OWNER_FULL_CONTROL,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add a separate test case for each type of BucketAccessControl, so we can ensure that all types are correctly transformed by toKebabCase()?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added the test.

});

// THEN
Expand All @@ -340,10 +341,47 @@ test('system metadata is correctly transformed', () => {
'expires': expiration.date.toUTCString(),
'sse-c-copy-source': 'rot13',
'website-redirect': 'example',
'acl': 'bucket-owner-full-control',
},
});
});

// type checking structure that forces to update it if BucketAccessControl changes
// see `--acl` here: https://docs.aws.amazon.com/cli/latest/reference/s3/sync.html
const accessControlMap: Record<s3.BucketAccessControl, string> = {
[s3.BucketAccessControl.PRIVATE]: 'private',
[s3.BucketAccessControl.PUBLIC_READ]: 'public-read',
[s3.BucketAccessControl.PUBLIC_READ_WRITE]: 'public-read-write',
[s3.BucketAccessControl.AUTHENTICATED_READ]: 'authenticated-read',
[s3.BucketAccessControl.AWS_EXEC_READ]: 'aws-exec-read',
[s3.BucketAccessControl.BUCKET_OWNER_READ]: 'bucket-owner-read',
[s3.BucketAccessControl.BUCKET_OWNER_FULL_CONTROL]: 'bucket-owner-full-control',
[s3.BucketAccessControl.LOG_DELIVERY_WRITE]: 'log-delivery-write',
};

test.each(Object.entries(accessControlMap) as [s3.BucketAccessControl, string][])(
'system metadata acl %s is correctly transformed',
(accessControl, systemMetadataKeyword) => {
// GIVEN
const stack = new cdk.Stack();
const bucket = new s3.Bucket(stack, 'Dest');

// WHEN
new s3deploy.BucketDeployment(stack, 'Deploy', {
sources: [s3deploy.Source.asset(path.join(__dirname, 'my-website.zip'))],
destinationBucket: bucket,
accessControl: accessControl,
});

// THEN
expect(stack).toHaveResource('Custom::CDKBucketDeployment', {
SystemMetadata: {
acl: systemMetadataKeyword,
},
});
},
);

test('expires type has correct values', () => {
expect(cdk.Expiration.atDate(new Date('Sun, 26 Jan 2020 00:53:20 GMT')).date.toUTCString()).toEqual('Sun, 26 Jan 2020 00:53:20 GMT');
expect(cdk.Expiration.atTimestamp(1580000000000).date.toUTCString()).toEqual('Sun, 26 Jan 2020 00:53:20 GMT');
Expand Down