Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RFC 1: Faster Development Iterations (aka "CDK Watch") #335

Merged
merged 9 commits into from
Jul 15, 2021
Merged

RFC 1: Faster Development Iterations (aka "CDK Watch") #335

Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
dc74db4
CDK RFC for `update`
May 18, 2021
8f0d98c
Address eladb's comments on scope.
Jun 14, 2021
eb0bb40
Update post-sync w/ Elad
Jun 22, 2021
8e01a68
faq: address deployment safety
Jun 23, 2021
2cf7639
Typo fix.
offbyone Jun 24, 2021
db10d3d
Updates for PR review
Jun 24, 2021
736a7a9
Expanded the implementation plan
Jul 6, 2021
86ac9f8
Apply suggestions from code review
offbyone Jul 12, 2021
1d1e562
Updated in response to PR feedback
Jul 12, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
131 changes: 57 additions & 74 deletions text/0001-cdk-update.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ rfc pr: [#335](https://github.com/aws/aws-cdk-rfcs/pull/335)
tracking issue: https://github.com/aws/aws-cdk-rfcs/issues/1
---

# Faster personal CDK deployments
# CDK Accelerate

Shorten the development iteration speed -- the "edit-compile-test" loop -- for
CDK applications.
Expand All @@ -12,16 +12,17 @@ CDK applications.

### CHANGELOG

- feat(cli): implement `cdk update` and `cdk update --watch` for asset updates
- feat(cli): implement `cdk deploy --accelerated` and
offbyone marked this conversation as resolved.
Show resolved Hide resolved
`cdk deploy --accelerated --watch`
offbyone marked this conversation as resolved.
Show resolved Hide resolved

### README
offbyone marked this conversation as resolved.
Show resolved Hide resolved

The `cdk update` command accelerates the edit-compile-test loop for your CDK
application during development by inspecting the assets and stack resources in
your application, identifying those that can be updated in-place without a full
CloudFormation stack udpate, and doing so using AWS service APIs directly. It
can either do this as an interactive command or by running continuously and
monitoring the input files.
The `cdk deploy --accelerated` ("CDK accelerate") command accelerates the
edit-compile-test loop for your CDK application during development by inspecting
the assets and stack resources in your application, identifying those that can
be updated in-place without a full CloudFormation stack udpate, and doing so
offbyone marked this conversation as resolved.
Show resolved Hide resolved
using AWS service APIs directly. It can either do this as an interactive command
or by running continuously and monitoring the input files.

For supported construct types (see list, below) the update command will identify
assets whose content has changed since the last time the CDK application was
offbyone marked this conversation as resolved.
Show resolved Hide resolved
Expand All @@ -35,35 +36,19 @@ contains resources supported by the update process. At this time, only Lambda
Functions and ECS Services are supported, however more resource types will be
added to this list in the future.

At a high level, `update` operates by reading the current state of the stack,
performing a stack diff on it, and if the stack is different only in resources
supported by update plugins, running those in lieu of a full deployment.

#### Preparing Your Stack

In order to ensure that update is not invoked against production stacks, you
must explicitly designate your stack as being updateable in your application.
This is a stack-level property, set in the `StackProps` parameter to the stack
constructor:

```typescript
const myDevelopmentStack = new ApplicationStack(app, "StackNameForDev", {
...
interactiveUpdate: true,
});
```

The `interactiveUpdate` property is optional, and when present and `true` the
`cdk update` functionality will be executed on the stack.
At a high level, `accelerate` operates by reading the current state of the
offbyone marked this conversation as resolved.
Show resolved Hide resolved
stack, performing a stack diff on it, and if the stack is different only in
resources supported by update plugins, running those in lieu of a full
deployment.

#### Running Update
offbyone marked this conversation as resolved.
Show resolved Hide resolved

To run a one-shot update, you can invoke the `cdk update` command on some or all
of your stacks:
To run a one-shot update, you can invoke the `cdk deploy --accelerated` command
offbyone marked this conversation as resolved.
Show resolved Hide resolved
on some or all of your stacks:

```
$ cdk update ApplicationStack
Checking stack ApplicationStack for possible asset update
$ cdk deploy --accelerated ApplicationStack
Checking stack ApplicationStack for possible accelerated update
* LambdaFunction[AppFunction]: File asset changed
ApplicationStack can be updated
1/1 stacks can be updated:
Expand All @@ -76,10 +61,10 @@ If the update is to an attribute of the stack that cannot be updated, the
command will offer to perform a full deployment:

```
$ cdk update ApplicationStack
Checking stack ApplicationStack for possible asset update
$ cdk deploy --accelerated ApplicationStack
Checking stack ApplicationStack for possible accelerated update
* LambdaFunction[Appfunction]: Function data changed
ApplicationStack can *not* be rapidly updated.
ApplicationStack has changes that can not be rapidly updated.
Perform a full deployment of ApplicationStack? [Y/n]: Y
offbyone marked this conversation as resolved.
Show resolved Hide resolved
Deploying ApplicationStack
...
Expand All @@ -90,11 +75,11 @@ When multiple stacks are provided, update only occurs if all pending changes are
suitable for update. Otherwise, a full deployment is done:

```
$ cdk update ApplicationStack DataStack
Checking stack ApplicationStack for possible asset update
$ cdk deploy --accelerated ApplicationStack DataStack
Checking stack ApplicationStack for possible accelerated update
* LambdaFunction[Appfunction]: Function data changed
ApplicationStack does not support update.
Checking stack DataStack for possible asset update
ApplicationStack has changes that can not be rapidly updated.
Checking stack DataStack for possible accelerated update
* LambdaFunction[StreamFunction]: File asset changed
DataStack can be updated
1/2 stacks can be updated: Perform a full deployment? [Y/n]? Y
Expand All @@ -105,16 +90,16 @@ Deploying DataStack
Done!
```

In addition to running in a one shot mode, the `cdk update` command also has a
`--watch` command line option that enable it to monitor the assets on disk and
perform an update when they change.
In addition to running in a one shot mode, the `cdk deploy --accelerated`
offbyone marked this conversation as resolved.
Show resolved Hide resolved
command also has a `--watch` command line option that enable it to monitor the
assets on disk and perform an update when they change.

```
$ cdk update --watch ApplicationStack DataStack
Checking stack ApplicationStack for possible asset update
$ cdk deploy --accelerated --watch ApplicationStack DataStack
Checking stack ApplicationStack for possible accelerated update
* LambdaFunction[Appfunction]: No Changes
ApplicationStack can *not* be updated
Checking stack DataStack for possible asset update
Checking stack DataStack for possible accelerated update
* LambdaFunction[StreamFunction]: No Changes

Watching stack inputs for changes:
Expand Down Expand Up @@ -145,24 +130,25 @@ images.

### PRESS RELEASE
offbyone marked this conversation as resolved.
Show resolved Hide resolved

DATE - AWS announces the `cdk update` command for the AWS CDK toolkit. The
`update` command allows CDK application developers to rapidly update code within
their CDK application when no other AWS resources are changing, bypassing the
more correct, but time consuming, stack update procedure.
DATE - AWS announces the `cdk deploy --accelerated` command for the AWS CDK
toolkit. The `accelerate` command allows CDK application developers to rapidly
update code within their CDK application when no other AWS resources are
changing, bypassing the more correct, but time consuming, stack update
procedure.

The CDK toolkit uses CloudFormation under the hood to manage changes to AWS
resources, which safely updates resources. This update process is slow, however,
and during development it performs safety checks that add overhead to developer
workflows. Those delays are interposed between every change a developer makes,
and over the course of a product may add hours to the total development time.

The `cdk update` command provides a way to bypass those checks when working in
development, directly updating the code in Lambda functions and ECS services
without a full CDK deployment. When the `update` command can identify that all
changes in the stack have shortcut support, it will directly publish the assets
to AWS and then modify the underlying CDK resources directly.
The `cdk deploy --accelerated` command provides a way to bypass those checks
when working in development, directly updating the code in Lambda functions and
ECS services without a full CDK deployment. When the `accelerate` command can
identify that all changes in the stack have shortcut support, it will directly
publish the assets to AWS and then modify the underlying CDK resources directly.

> Using cdk update dramatically improves our development iteration speed for
> Using CDK accelerate dramatically improves our development iteration speed for
> Lambda functions. It cuts down the change-deploy-test cycle to seconds, where
> it was minutes before with a full cdk deploy each time. Unlike some other
> solutions, it actually updates our function running on AWS, which means there
Expand All @@ -171,14 +157,15 @@ to AWS and then modify the underlying CDK resources directly.

Any CDK users who are deploying code to Lambda functions or containers on ECS or
Fargate can take advantage of this tool today, by configuring their stacks for
interactive update, and then using `cdk update` instead of `cdk deploy`.
interactive update, and then using `cdk deploy --accelerated` instead of
`cdk deploy`.

## FAQ

### What are we launching today?

The `cdk update` and `cdk update --watch` commands, with support for rapid
update of Lambda function code and Fargate images.
The `cdk deploy --accelerated` and `cdk deploy --accelerated --watch` commands,
with support for rapid update of Lambda function code and Fargate images.
offbyone marked this conversation as resolved.
Show resolved Hide resolved

### Why should I use this feature?

Expand Down Expand Up @@ -208,7 +195,7 @@ since each code change currently requires a CloudFormation stack update to apply
new code, _or_ manually introducing drift to the application by inspecting the
stack and manipulating the resources directly.

The CDK update tool will allow the CDK to handle the resource changes rather
The CDK accelerate tool will allow the CDK to handle the resource changes rather
than this manual process, introducing some implicit safety and reducing the
manual labor of the update.

Expand All @@ -219,25 +206,17 @@ to shortcircuit the safe CloudFormation deployment process. If a user runs CDK
update on their production stack, it can perform updates without adequate stack
update safety checks. Releasing a public tool with instant update capability
into the CDK may not be the right way to make this functionality public. To
mitigate this, `update` only runs on stacks that have explicitly opted in to the
capability.
mitigate this, `accelerate` only runs on stacks that have explicitly opted in to
offbyone marked this conversation as resolved.
Show resolved Hide resolved
the capability.

### What changes are required to enable this change?

This RFC requires three changes:

1. An implementation of the `cdk update` command in the CDK toolkit that can
examine the Application stacks for updates that can be applied and apply
them.
2. Adding the `interactiveUpdate` flag to the `StackProps` object, allowing a
stack to be configured for interactive update. This property will default to
`false`
This RFC requires two changes:

**note:** I chose in-stack markup instead of a `cdk.json` flag so that if
there is an L3 construct for a personal development stack (or a library that
creates them) then the attribute can be set there instead.

3. An expansion of the CDK asset model to include optional construction process
1. An implementation of the `cdk deploy --accelerated` command in the CDK
toolkit that can examine the Application stacks for updates that can be
applied and apply them.
2. An expansion of the CDK asset model to include optional construction process
metadata and construction input metadata. This is required to allow CDK to
identify input changes to the asset, reconstruct it from the defined
processes, and then publish it to the necessary AWS service.
Expand Down Expand Up @@ -270,6 +249,10 @@ We also considered introspecting the CDK model ourselves, but concluded that
there was little value in reinventing the wheel, so to speak, when CDK already
had all of the information we'd need to deliver this.

For identifying stacks that are subject to accelerated deployment, we are
considering defining a full "Personal Development Stack" model, possibly based
off of information in the CDK `App` context.

### What is the high level implementation plan?

We will start from the prototype CDK update command that identifies the Lambda
Expand Down