Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(decdk) prototype for declarative CDK (decdk) #1618

Merged
merged 47 commits into from
Feb 13, 2019
Merged

Conversation

rix0rrr
Copy link
Contributor

@rix0rrr rix0rrr commented Jan 26, 2019

A prototype for a tool that reads CloudFormation-like JSON/YAML templates which can contain both normal CloudFormation resources (AWS::S3::Bucket) and also reference AWS CDK resources (@aws-cdk/aws-s3.Bucket).

See README for details.


Pull Request Checklist

  • Testing
    • Unit test added
    • CLI change?: manually run integration tests and paste output as a PR comment
    • cdk-init template change?: coordinated update of integration tests with team
  • Docs
    • jsdocs: All public APIs documented
    • README: README and/or documentation topic updated
  • Title and Description
    • Change type: title prefixed with fix, feat will appear in changelog
    • Title: use lower-case and doesn't end with a period
    • Breaking?: last paragraph: "BREAKING CHANGE: <describe what changed + link for details>"
    • Issues: Indicate issues fixed via: "Fixes #xxx" or "Closes #xxx"
  • Sensitive Modules (requires 2 PR approvers)
    • IAM Policy Document (in @aws-cdk/aws-iam)
    • EC2 Security Groups and ACLs (in @aws-cdk/aws-ec2)
    • Grant APIs (only if not based on official documentation with a reference)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license.

@rix0rrr rix0rrr requested a review from a team as a code owner January 26, 2019 18:03
Elad Ben-Israel added 16 commits January 26, 2019 20:30
turn decdk to a self-contained transformer which depends on all cdk
libraries to simplify usage.

theoretically we would have been able to split decdk and the dependency
closure, but keeping them monolithic right now is easier to work with
and takes care of some issues related to resolving modules
in client packages (e.g. we need to resolve transitive modules and
that's annoying).
- support enums
- inherit interface members
- use normal jsii FQNs for package names to keep things simple
if an interface has non-serializable properties but they are
optional, we can technically allow users to use the construct
with the default value. this dramatically expands the set of
available deconstructs.
- performs type checking
- allows deserializing enums
- in the future will allow deserializing IBucket-like objects
-
if we see an Fn::GetAtt, we replace it with a synth-time lazy value
which will resolve according to what is referenced.

if GetAtt references a high level construct, the resolved value will
be the value returned by a named property. otherwise, we just retain
the original Fn::GetAtt.

at the moment, this is only supported for strings.
a new "resourceForPath(path)" method will get or create all
the resources leading to a certain path in the api model.

this can be used to easily define "routes" in the api and
reuse the nodes in the tree.

includes a refactor of how the root resource and normal resources
share implementation (through an abstract base class, dah!)
the ApiEventSource allows easily adding APIGW routes to a Lambda.
upon first call, a new REST API resource will be created, and then
routes will be added to it for each ApiEventSource associated 
with the lambda.
initial implementation for a declarative surface for serverless
functions, which is based on the SAM spec. the main difference
between this API and the Lambda API is that this allows defining
event sources declaratively when the function is initialized.
- ECS/Fargate: one resource that synthesizes 38
- Queue with KMS encryption with auto-created key
- SAM API events + dynamo table + GetAtt in Outputs
- SAM with Queues: shows GetAtt for both resource and construct

A script "./synth" instead of cdk.json
If a construct has a property that references another
construct, users can now use {Ref} to reference a construct
of that type that was defined in the same template.

The schema will only allow { Ref: ID }
Classes that have static methods/properties that return
a type that matches the class type can be considered "enum-like"
because they can be used like so:

    lambda.Runtime.NodeJS10

or:

    lambda.Code.asset('path')

We are now deconstructing these to either a string (for the property
case) or an object where the key is the method name and the value
is an map of parameter names to values.

This is actually something we can probably embed in the jsii spec,
as it is a common pattern.
To enable deconstruction, turn ContainerImage to an enum-like class. This means that the "image" property needs to accept a concrete class instead of an interface. Seems like this won't hurt the model in any way.
in order to be able to deconstruct this, we expect ctors to have only three arguments: scope,id,props.

split ContainerDefinitionOptions from ContainerDefinitionProps to allow
reusing them in `addContainer`.

also, move the logic of "linking" the container to the task definition
into the ContainerDefinition class, so that "addContainer" is pure
sugar and doesn't have special behavior. otherwise, we won't be able
to use it declaratively.
- ecs
- lambda
- pipelines(!)
Copy link
Contributor

@sam-goodwin sam-goodwin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is beautiful.


export interface Event {
type: EventType;
properties: DynamoEvent | SnsEvent | SqsEvent | ApiEvent;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're missing Kinesis.

this.handler.addEventSource(new events.ApiEventSource(apiEvent.method, apiEvent.path));
break;

default:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kinesis, Dynamo?


## Open issues

- [ ] Do we need a "ref" attribute for allowing referencing resources?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the story for L1 and raw CFN?

Elad Ben-Israel added 4 commits January 30, 2019 14:21
A partition key is a required property of DynamoDB.

BREAKING CHANGE: partitionKey is now required when defining a DynamoDB table. the method "addPartitionKey" has been removed.
add SimpleTable, which is a DynamoDB table with a default partition
key of "ID: string".

SimpleTable extends from Table and support all features.
If a property accepts an interface type, deCDK will identify all types
that implement it and will formulate a JSON schema that allows users
to instantiate these types.

For example lambda IEventSource:

    "events": [
      { 
        "DynamoEventSource": { 
          "table": { "Ref": "Table" }, 
          "props": { "startingPosition": "TrimHorizon" } 
        } 
      },
      { "ApiEventSource": { "method": "GET", "path": "/hello" } },
      { "ApiEventSource": { "method": "POST", "path": "/hello" } },
      { "SnsEventSource": { "topic": { "Ref": "MyTopic" } } }
    ]
@eladb eladb self-assigned this Feb 4, 2019
@eladb eladb changed the title [WIP] Rewrite of declarative CDK [WIP] deCDK Feb 4, 2019
@eladb eladb changed the title [WIP] deCDK [WIP] deCDK - Declarative CDK Feb 4, 2019
* an `Fn::GetAtt`.
*/
function deconstructGetAtt(stack: cdk.Stack, id: string, attribute: string) {
new cdk.Token(() => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return?

@eladb eladb changed the title [WIP] deCDK - Declarative CDK deCDK - Declarative CDK - Prototype Feb 11, 2019
eladb pushed a commit that referenced this pull request Feb 12, 2019
Minor changes to the ECS APIs so that they can be instantiated via a declarative
template (see deCDK #1618).

Merge `IContainerImage` and the base `ContainerImage` into an abstract class
so it's now an "enum-like" class with static methods. It also improves discoverability
for all other users and more aligned with how other constructs expose union types
(e.g. `lambda.Code`).

Normalize the ctor of `ContainerDefinition` to "scope, id, props" so that it can
be instantiated as a deCDK resource.
eladb pushed a commit that referenced this pull request Feb 13, 2019
Minor changes to the ECS APIs so that they can be instantiated via a declarative
template (see deCDK #1618).

Merge `IContainerImage` and the base `ContainerImage` into an abstract class
so it's now an "enum-like" class with static methods. It also improves discoverability
for all other users and more aligned with how other constructs expose union types
(e.g. `lambda.Code`).

Normalize the ctor of `ContainerDefinition` to "scope, id, props" so that it can
be instantiated as a deCDK resource.
tools/decdk/README.md Outdated Show resolved Hide resolved
tools/decdk/README.md Outdated Show resolved Hide resolved
tools/decdk/README.md Outdated Show resolved Hide resolved
tools/decdk/README.md Outdated Show resolved Hide resolved
tools/decdk/deps.js Outdated Show resolved Hide resolved
Copy link
Contributor

@eladb eladb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving on behalf of @rix0rrr because he is the submitter

@eladb eladb changed the title deCDK - Declarative CDK - Prototype feat(decdk) prototype for declarative CDK (decdk) Feb 13, 2019
@eladb eladb merged commit 8713ac6 into master Feb 13, 2019
@eladb eladb deleted the offsite/declarative branch February 13, 2019 13:59
@fulghum fulghum added effort/medium Medium work item – several days of effort effort/small Small work item – less than a day of effort and removed effort/medium Medium work item – several days of effort labels Feb 18, 2019
@NGL321 NGL321 added the contribution/core This is a PR that came from AWS. label Sep 27, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contribution/core This is a PR that came from AWS. effort/small Small work item – less than a day of effort
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants