Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(aws-s3-deployment): bucket deployments #971

Merged
merged 21 commits into from
Oct 19, 2018
Merged

Conversation

eladb
Copy link
Contributor

@eladb eladb commented Oct 19, 2018

This new library allows deploying contents to S3 buckets as part of a CDK app from a .zip file in another bucket or from a local asset (directory or a .zip file).

This is how it's used:

new BucketDeployment(this, 'DeployMe', {
  source: Source.asset('/local/directory'),
  destinationBucket: dest
});

"source" can either be a local directory, a local zip file or another
bucket+key.

This library is backed by a custom CloudFormation resource which uses the
AWS CLI to invoke "aws s3 sync".

By default, when the resource is deleted, the contents of the destination are deleted. The RetainOnDelete option can be used to disable this behavior.

Fixes #952.

Future plans:

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license.

This new library allows deploying contents to S3 buckets as part of a CDK app from a .zip file in another bucket or from a local asset (directory or a .zip file).

This is how it's used:

    new BucketDeployment(this, 'DeployMe', {
      source: Source.asset('/local/directory'),
      destinationBucket: dest
    });

"source" can either be a local directory, a local zip file or another
bucket+key.

This library is backed by a custom CloudFormation resource which uses the
AWS CLI to invoke "aws s3 sync". 

By default, when the resource is deleted, the contents of the destination are deleted. The `RetainOnDelete` option can be used to disable this behavior.

Fixes #952.


Future plans:

- The current version will always use "--delete" which means destination files will be deleted if they don't exist on the source (will be addressed in #953).
- Another mode of operation that we plan to add is blue/green deployments (#954).

�
logger.info("| s3_dest: %s" % s3_dest)

# delete or create/update
if request_type == "Delete":
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aaaaactually you probably also need to do this on Update; it might be that your resource got pointed to a different bucket, and so you probably want to delete from the old one (OldResourceProperties)).

Sorry for not thinking of this earlier.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch!

I guess the safest thing to do is if the destination bucket+prefix is different from the new bucket+prefix, we first delete the old one and then sync to the new one. Makes sense?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, an Update should probably be treated as Delete+Create

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't forget that the old bucket is in a different argument (OldResourceProperties instead of ResourceProperties)

@eladb eladb merged commit 84d6876 into master Oct 19, 2018
@eladb eladb deleted the benisrae/bucket-deployments branch October 19, 2018 13:04
eladb pushed a commit that referenced this pull request Oct 19, 2018
### Highlights

 - __A new construct library for AWS Step Functions__
   ([docs](https://github.com/awslabs/aws-cdk/blob/master/packages/%40aws-cdk/aws-stepfunctions/README.md)).
   The library provides rich APIs for modeling state machines by exposing a
   programmatic interface for [Amazon State
   Language](https://docs.aws.amazon.com/step-functions/latest/dg/concepts-amazon-states-language.html).
 - __A new construct library for Amazon S3 bucket deployments__
   ([docs](https://github.com/awslabs/aws-cdk/blob/master/packages/%40aws-cdk/aws-s3-deployment/README.md)).
   You can use now automatically populate an S3 Bucket from a .zip file or a
   local directory. This is a building block for end-to-end support for static
   websites in the AWS CDK.

### Bug Fixes

* **aws-apigateway:** make LambdaRestApi proxy by default ([#963](#963)) ([a5f5e2c](a5f5e2c)), closes [#959](#959)
* **aws-cdk:** Allow use of assumed roles behind a proxy ([#898](#898)) ([f2b1048](f2b1048))
* **aws-cdk:** Auto-delete stacks that failed creating before new attempt ([#917](#917)) ([2af8309](2af8309))
* **aws-cloudfront:** expose distributionId ([#938](#938)) ([f58d98c](f58d98c))
* **aws-dynamodb:** don't emit empty array properties ([#909](#909)) ([841975a](841975a))
* **docs:** use ..code to display file structure in "writing constructs" ([#935](#935)) ([b743362](b743362))

### Features

* **assets:** isZipArchive indicates if this is a zip asset ([#944](#944)) ([65190f9](65190f9))
* **aws-cdk:** deploy supports CloudFormation Role ([#940](#940)) ([393be6f](393be6f)), closes [#735](#735)
* **aws-cloudformation:** allow specifying custom resource type ([#943](#943)) ([9de3a84](9de3a84))
* **aws-cloudformation:** correctly handle the templateConfiguration property in the CreateUpdateStack Pipeline Action. ([#923](#923)) ([d251a46](d251a46))
* **aws-cloudfront:** add support for "webAclId" ([#969](#969)) ([3ec9d76](3ec9d76))
* **aws-codedeploy:** add auto rollback configuration to server Deployment Group. ([#925](#925)) ([7ee91cf](7ee91cf))
* **aws-codedeploy:** add instance tag filter support for server Deployment Groups. ([#824](#824)) ([e6e8c51](e6e8c51))
* **aws-codedeploy:** add support for setting CloudWatch alarms on a server Deployment Group. ([#926](#926)) ([27b26b1](27b26b1))
* add support for Step Functions ([#827](#827)) ([81b533c](81b533c))
* **aws-lambda:** add grantInvoke() method ([#962](#962)) ([1ee8135](1ee8135)), closes [#961](#961)
* **aws-lambda:** improvements to the code and runtime APIs ([#945](#945)) ([36f29b6](36f29b6)), closes [#902](#902) [#188](#188) [#947](#947) [#947](#947) [#664](#664)
* **aws-logs:** extractMetric() returns Metric object ([#939](#939)) ([5558fff](5558fff)), closes [#850](#850)
* **aws-s3:** initial support for website hosting ([#946](#946)) ([2d3661c](2d3661c))
* **aws-s3-deployment:** bucket deployments ([#971](#971)) ([84d6876](84d6876)), closes [#952](#952) [#953](#953) [#954](#954)
* **docs:** added link to CloudFormation concepts ([#934](#934)) ([666bbba](666bbba))

### BREAKING CHANGES

* **aws-apigateway:** specifying a path no longer works. If you used to
provide a '/', remove it. Otherwise, you will have to supply `proxy: false`
and construct more complex resource paths yourself.
* **aws-lambda:** The construct `lambda.InlineJavaScriptLambda` is no longer supported. Use `lambda.Code.inline` instead; `lambda.Runtime.NodeJS43Edge` runtime is removed. CloudFront docs [stipulate](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-requirements-limits.html#lambda-requirements-lambda-function-configuration) that you should use node6.10 or node8.10. It is always possible to use any value by instantiating a `lambda.Runtime` object.
@eladb eladb mentioned this pull request Oct 19, 2018
eladb pushed a commit that referenced this pull request Oct 19, 2018
### Highlights

 - __A new construct library for AWS Step Functions__
   ([docs](https://github.com/awslabs/aws-cdk/blob/master/packages/%40aws-cdk/aws-stepfunctions/README.md)).
   The library provides rich APIs for modeling state machines by exposing a
   programmatic interface for [Amazon State
   Language](https://docs.aws.amazon.com/step-functions/latest/dg/concepts-amazon-states-language.html).
 - __A new construct library for Amazon S3 bucket deployments__
   ([docs](https://github.com/awslabs/aws-cdk/blob/master/packages/%40aws-cdk/aws-s3-deployment/README.md)).
   You can use now automatically populate an S3 Bucket from a .zip file or a
   local directory. This is a building block for end-to-end support for static
   websites in the AWS CDK.

### Bug Fixes

* **aws-apigateway:** make LambdaRestApi proxy by default ([#963](#963)) ([a5f5e2c](a5f5e2c)), closes [#959](#959)
* **aws-cdk:** Allow use of assumed roles behind a proxy ([#898](#898)) ([f2b1048](f2b1048))
* **aws-cdk:** Auto-delete stacks that failed creating before new attempt ([#917](#917)) ([2af8309](2af8309))
* **aws-cloudfront:** expose distributionId ([#938](#938)) ([f58d98c](f58d98c))
* **aws-dynamodb:** don't emit empty array properties ([#909](#909)) ([841975a](841975a))
* **docs:** use ..code to display file structure in "writing constructs" ([#935](#935)) ([b743362](b743362))

### Features

* **assets:** isZipArchive indicates if this is a zip asset ([#944](#944)) ([65190f9](65190f9))
* **aws-cdk:** deploy supports CloudFormation Role ([#940](#940)) ([393be6f](393be6f)), closes [#735](#735)
* **aws-cloudformation:** allow specifying custom resource type ([#943](#943)) ([9de3a84](9de3a84))
* **aws-cloudformation:** correctly handle the templateConfiguration property in the CreateUpdateStack Pipeline Action. ([#923](#923)) ([d251a46](d251a46))
* **aws-cloudfront:** add support for "webAclId" ([#969](#969)) ([3ec9d76](3ec9d76))
* **aws-codedeploy:** add auto rollback configuration to server Deployment Group. ([#925](#925)) ([7ee91cf](7ee91cf))
* **aws-codedeploy:** add instance tag filter support for server Deployment Groups. ([#824](#824)) ([e6e8c51](e6e8c51))
* **aws-codedeploy:** add support for setting CloudWatch alarms on a server Deployment Group. ([#926](#926)) ([27b26b1](27b26b1))
* add support for Step Functions ([#827](#827)) ([81b533c](81b533c))
* **aws-lambda:** add grantInvoke() method ([#962](#962)) ([1ee8135](1ee8135)), closes [#961](#961)
* **aws-lambda:** improvements to the code and runtime APIs ([#945](#945)) ([36f29b6](36f29b6)), closes [#902](#902) [#188](#188) [#947](#947) [#947](#947) [#664](#664)
* **aws-logs:** extractMetric() returns Metric object ([#939](#939)) ([5558fff](5558fff)), closes [#850](#850)
* **aws-s3:** initial support for website hosting ([#946](#946)) ([2d3661c](2d3661c))
* **aws-s3-deployment:** bucket deployments ([#971](#971)) ([84d6876](84d6876)), closes [#952](#952) [#953](#953) [#954](#954)
* **docs:** added link to CloudFormation concepts ([#934](#934)) ([666bbba](666bbba))

### BREAKING CHANGES

* **aws-apigateway:** specifying a path no longer works. If you used to
provide a '/', remove it. Otherwise, you will have to supply `proxy: false`
and construct more complex resource paths yourself.
* **aws-lambda:** The construct `lambda.InlineJavaScriptLambda` is no longer supported. Use `lambda.Code.inline` instead; `lambda.Runtime.NodeJS43Edge` runtime is removed. CloudFront docs [stipulate](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-requirements-limits.html#lambda-requirements-lambda-function-configuration) that you should use node6.10 or node8.10. It is always possible to use any value by instantiating a `lambda.Runtime` object.
@eladb eladb mentioned this pull request Nov 7, 2018
@NGL321 NGL321 added the contribution/core This is a PR that came from AWS. label Sep 27, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contribution/core This is a PR that came from AWS.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

s3-deployment: allow deletion of destination contents when resource is deleted
3 participants