From dc41934031a3cc704843dce72c6cf41f49b14991 Mon Sep 17 00:00:00 2001 From: Shiv Lakshminarayan Date: Thu, 12 Sep 2019 07:14:04 -0700 Subject: [PATCH 1/9] rfc: publish cdk construct tree --- design/construct-tree.md | 223 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 223 insertions(+) create mode 100644 design/construct-tree.md diff --git a/design/construct-tree.md b/design/construct-tree.md new file mode 100644 index 0000000000000..a8e63bf65ecc5 --- /dev/null +++ b/design/construct-tree.md @@ -0,0 +1,223 @@ +## RFC: PUBLISH CDK CONSTRUCT TREE + +## Overview + +This is a strategy for publishing the construct tree which represents a CDK app as a part of the Cloud Assembly that the CDK generates. + +***Goal*: Expose the** **`construct tree`** **for CDK Apps as an artifact of the Cloud Assembly** + +This proposal details the motivation behind the goal, requirements, specification of the construct tree, and the proposed design. The scope of the design is focused on defining what the construct tree should include and the approach to produce it as an output. + +Prototype: https://github.com/aws/aws-cdk/tree/prototypes/cdk-look + +## Background + +ALL CDK applications are composed of constructs, which are the basic building blocks of AWS CDK applications. They encapsulate everything that underlying providers such as AWS CloudFormation need to create cloud components. + +A construct can represent a single resource, such as an Amazon Simple Storage Service (Amazon S3) bucket, or it can represent a higher-level component consisting of multiple AWS CDK resources. Examples of such components include a worker queue with its associated compute capacity, a cron job with monitoring resources and a dashboard, or even an entire app spanning multiple AWS accounts and regions. + +The CDK CLI is the primary mechanism through which developers currently interact with their AWS CDK applications. It supports various operations throughout the lifecycle of application development from from the initialization of a CDK app from a template to the deployment and destruction of the AWS CloudFormation stacks. + +## Motivation + +Developers author their CDK applications by leveraging higher-level intent based APIs offered through the [AWS Construct Library](https://docs.aws.amazon.com/cdk/api/latest/docs/aws-construct-library.html). They use CLI commands such as `cdk synth` to produce the cloud assembly and `cdk deploy` to deploy it to AWS CloudFormation. When deployments fail, they often have to drop into the AWS CloudFormation templates that the CDK generated or log into the AWS console to trace their issues. + +As an example, the following CDK application for an Application Load Balancer includes constructs from Auto Scaling, VPC, and Application Load Balancing. + +```ts + const vpc = new ec2.Vpc(this, 'VPC'); + + const asg = new autoscaling.AutoScalingGroup(this, 'ASG', { + vpc, + instanceType: ec2.InstanceType.of(ec2.InstanceClass.T2, ec2.InstanceSize.MICRO), + machineImage: new ec2.AmazonLinuxImage(), + }); + + const lb = new elbv2.ApplicationLoadBalancer(this, 'LB', { + vpc, + internetFacing: true + }); + + const listener = lb.addListener('Listener', { + port: 80, + }); + + listener.addTargets('Target', { + port: 80, + targets: [asg] + }); + + listener.connections.allowDefaultPortFromAnyIpv4('Open to the world'); + + asg.scaleOnRequestCount('AModestLoad', { + targetRequestsPerSecond: 1 + }); +``` + + +Running `cdk synth` on this application produces ~800 lines of CloudFormation template and represents over 35 AWS resources spanning 5 services. The generated template is part of the cloud assembly and will be used subsequently by deploy and is staged in the `cdk.out` directory. This showcases the power of the CDK in the ability to define high level abstractions of infrastructure in familiar languages that can be reused. + +Continuing with this example, inspecting the the cloud assembly’s CloudFormation template doesn’t help gain any insight when behavior does not match expectations. + +In this scenario, these are some details that are not intuitive without diving deeper: + +> What resources does the AutoScalingGroup construct contain? the VPC construct? + +> What properties were the resources initialized with? Can they be changed? + +> Is the relationship between these constructs captured in the cloud assembly correctly? + + +When deployments fail, or provisioned resources don’t hold the properties that one might expect, developers can’t figure out what properties that resources such as `NatGateway, AutoScalingGroup, Subnets, Route Tables` were configured with without dropping into the generated CloudFormation template. It can take a good amount of clicking and digging before developers can figure out what happened and almost certainly needs them to get away from their favourite IDE and switch context to find out. + +The recurring theme is that intent based constructs are easy to get started with, take a lot of complexity away from the developer. The flip-side of this experience is that debugging and figuring out what the CDK did on your behalf can become challenging as applications come more complex. This problem will proliferate as more third-party constructs become available for consumption. + + Exposing the construct tree will enrich the developer experience for CDK application construction by exposing the construct tree explicitly. + + +Let’s take another look at an example of what a construct tree might look like: + +This is a Lambda function that has it’s handler code written as an asset: + +```ts + new lambda.Function(this, 'HelloWorldHandler', { + runtime: lambda.Runtime.NODE_JS_8_10, + code: lambda.Code.directory('lambda'), + handler: 'hello.handler' + }); +``` + +Ideally this simple Lambda code, will be represented by a construct tree that looks something like the following, with the ability to expand on constructs and navigate all the way down to their properties and values: + +```bash +├── App +│ ├── Stack 1 +│ │ ├── Lambda Function [L2 Construct](... properties ...) +| │ │ ├── IAM Role (policy information) +| │ │ ├── Asset Code (Lambda Handler, pointer to S3 location where Asset will reside) +| │ │ ├── Lambda Function (Function definition and properties) +``` + +## Requirements + +This list describes only the minimal set of requirements from this feature. After we release these building blocks, we will look into extending the model to meet other use cases that arise. + +1. **Format** - Publish underlying data as a `.json` file that contains the information required to render a construct tree view. The construct tree illustrated in the previous section is only a depiction of what a human readable(ish) view might look like +2. **Constructs** + 1. ***Nodes*** - CDK application constructs at the root and drill down to the constructs it contains (1+ cloud resources.) Initially, we will focus our attention on AWS resources, but the model is extensible to cloud components from any provider + 2. ***Metadata -*** Constructs will expose metadata which will include an array of objects associated with the construct. + 1. ***Properties*** - Higher level constructs (L2 and L3) will expose properties and values of the resources they contain. This will especially be useful for constructs that are opinionated and set defaults on behalf of developers + 2. ***Errors/Warnings*** - Errors and warnings that are produced during synthesis indicating validation failures, deprecation notices, guidance, etc will be included in the tree. These are emitted at the construct level although the message may point towards a specific property. + 3. **Assets** - Assets information will be included in the construct tree. Assets represent actions that the CDK *will* take ahead of stack deployments (i.e. S3 assets are zipped and uploaded directly, Docker images are uploaded to ECR). The asset metadata in the tree is the S3 key that an asset *would* have +3. **Local** - The construct tree is produced at synthesis time. Connectivity to AWS and CloudFormation should not be required to produce a tree view of a CDK application. + +## Approach + +At a high-level, we will render the `construct tree` through a construct tree data model and construct metadata interface that constructs will implement: + +### Construct Tree Data Model + +A feature will be added to the CDK core module so that any CDK application will produce an artifact called `tree.json` during calls to `cdk synth` and include it as an output to the cloud assembly in `cdk.out`. Developers will be able to opt-out if this functionality is not desired or needed through a flag in the CLI or set it as a preference. + +The construct tree will be a list of paths that are indexed into a flat map of constructs. The file structure itself represents the tree. This information will need to include the following to start rendering a flat map of constructs: + +### Construct properties + +|Property |Type |Required |Description | +|--- |--- |--- |--- | +|id |string |Required |id of the construct within the current scope | +|path |string |Required |full, absolute path of the construct within the tree | + +### Metadata Properties + +The following metadata properties will be included by the construct that produces the `tree.json` output. + +|Property |Type |Required |Description | +|--- |--- |--- |--- | +|sourceLocation |sourceLocation |Required |location in source code where the construct is defined | +|url |String |Required |array of metadata objects associated with this construct. | +|renderedView |Array |Not Required |constructs can fill in arbitrary. ClouFormation agnostic. CFN constructs could pre-populate it with the output of `this.toCloudFormation()` | +|children |Array |Not Required |All direct children of this construct. Array of the absolute paths of the constructs. Will be used to walk entire list of constructs | +|errors |Array |Not Required |array of errors associated with this construct. | +|warnings |Array |Not Required |array of warnings associated with this construct | +|description |string |Not Required |description of the construct | + +**TODO** - add concrete walkthrough example of a CDK application and the construct-tree flat map it would produce + +As a temporary placeholder, here's the implementation of the prototype construct referenced in the overview. It's a representation of the construct-tree of the [CDK Workshop](https://cdkworkshop.com/) application. + +```json +{ + "id": "App", + "path": "", + "children": [{ + "id": "CdkWorkshopStack", + "path": "CdkWorkshopStack", + "children": [{ + "id": "HelloHandler", + "path": "CdkWorkshopStack/HelloHandler", + "children": [{ + "id": "ServiceRole", + "path": "CdkWorkshopStack/HelloHandler/ServiceRole", + "children": [{ + "id": "Resource", + "path": "CdkWorkshopStack/HelloHandler/ServiceRole/Resource", + "metadata": { + "resourceType": "AWS::IAM::Role", + "logicalId": "HelloHandlerServiceRole11EF7C63" + }, + "links": [{ + "sourcePath": "CdkWorkshopStack/HelloHandler/Resource", + "targetPath": "CdkWorkshopStack/HelloHandler/ServiceRole/Resource", + "attribute": "Arn" + }] + }] + }, + ... +``` + +### Construct metadata + +We will also introduce an interface that constructs *can* implement to provide metadata such as properties, additional context, anything a construct would want to report back into the rendered view. Constructs will be able to supply information about themselves for inclusion into the context tree. + +The context tree provides the skeleton and the structure for rendering a tree-view of a CDK application. The proposed Interface will be added to `construct.ts` in the core library and implemented by `constructs` that have additional information to contribute to the construct tree : + +```ts +/*** + * Displayable information that a construct will contribute towards the context tree* + * such as dependencies, properties, links, documentation, etc + */ +export interface IDisplayable { + + // id of the construct within the current scope* + readonly id: string + + //array of metadata objects associated with this construct.* + readonly url: string + + // Other displayable metadata* + readonly metadata: [key: string]: string; + + /* method that supports extensibility on what can be displayed without + * adding more methods/properties on IDisplayable + */ + inspect(display: IDisplayer): void +} +``` + +## Unresolved Questions + +* Experience across all supported languages - nuances, considerations, etc +* Performance - `synth` can take longer to render with this added construct and additional + +## Future Possibilities + +We do not expect customers to have to interact directly with the outputs that the CDK produces as a part of the cloud assembly. Publishing the CDK construct tree as an output to the cloud assembly is intended to be a starting point to build on and create experiences based on it. + +### Tooling + +With a construct tree model in place and construct tree being published as a `.json` file, it opens the door to build tooling and utilities to enhance the application development experience so that developers can gain context and insight without having to leave their IDEs. Some of these features could include: + +**Diff** diff the construct tree against the resources that are deployed in the cloud. This capability would require cloud capability but we should always be able to render a construct tree without the need to connect. + +**Navigability** include the location in code where the construct is defined. This would start to pave the way towards building tooling and extensions that IDE's can leverage. \ No newline at end of file From ba9956f99680a68e411d83d94a8c56177c927194 Mon Sep 17 00:00:00 2001 From: Shiv Lakshminarayan Date: Thu, 12 Sep 2019 07:20:55 -0700 Subject: [PATCH 2/9] cleaning up some whitespace --- design/construct-tree.md | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/design/construct-tree.md b/design/construct-tree.md index a8e63bf65ecc5..3c77fb5e67798 100644 --- a/design/construct-tree.md +++ b/design/construct-tree.md @@ -2,7 +2,7 @@ ## Overview -This is a strategy for publishing the construct tree which represents a CDK app as a part of the Cloud Assembly that the CDK generates. +This is a strategy for publishing the construct tree which represents a CDK app as a part of the Cloud Assembly that the CDK generates. ***Goal*: Expose the** **`construct tree`** **for CDK Apps as an artifact of the Cloud Assembly** @@ -22,7 +22,7 @@ The CDK CLI is the primary mechanism through which developers currently interact Developers author their CDK applications by leveraging higher-level intent based APIs offered through the [AWS Construct Library](https://docs.aws.amazon.com/cdk/api/latest/docs/aws-construct-library.html). They use CLI commands such as `cdk synth` to produce the cloud assembly and `cdk deploy` to deploy it to AWS CloudFormation. When deployments fail, they often have to drop into the AWS CloudFormation templates that the CDK generated or log into the AWS console to trace their issues. -As an example, the following CDK application for an Application Load Balancer includes constructs from Auto Scaling, VPC, and Application Load Balancing. +As an example, the following CDK application for an Application Load Balancer includes constructs from Auto Scaling, VPC, and Application Load Balancing. ```ts const vpc = new ec2.Vpc(this, 'VPC'); @@ -57,17 +57,16 @@ As an example, the following CDK application for an Application Load Balancer in Running `cdk synth` on this application produces ~800 lines of CloudFormation template and represents over 35 AWS resources spanning 5 services. The generated template is part of the cloud assembly and will be used subsequently by deploy and is staged in the `cdk.out` directory. This showcases the power of the CDK in the ability to define high level abstractions of infrastructure in familiar languages that can be reused. -Continuing with this example, inspecting the the cloud assembly’s CloudFormation template doesn’t help gain any insight when behavior does not match expectations. +Continuing with this example, inspecting the the cloud assembly’s CloudFormation template doesn’t help gain any insight when behavior does not match expectations. In this scenario, these are some details that are not intuitive without diving deeper: -> What resources does the AutoScalingGroup construct contain? the VPC construct? +> What resources does the AutoScalingGroup construct contain? the VPC construct? > What properties were the resources initialized with? Can they be changed? > Is the relationship between these constructs captured in the cloud assembly correctly? - When deployments fail, or provisioned resources don’t hold the properties that one might expect, developers can’t figure out what properties that resources such as `NatGateway, AutoScalingGroup, Subnets, Route Tables` were configured with without dropping into the generated CloudFormation template. It can take a good amount of clicking and digging before developers can figure out what happened and almost certainly needs them to get away from their favourite IDE and switch context to find out. The recurring theme is that intent based constructs are easy to get started with, take a lot of complexity away from the developer. The flip-side of this experience is that debugging and figuring out what the CDK did on your behalf can become challenging as applications come more complex. This problem will proliferate as more third-party constructs become available for consumption. @@ -105,7 +104,7 @@ This list describes only the minimal set of requirements from this feature. Afte 1. **Format** - Publish underlying data as a `.json` file that contains the information required to render a construct tree view. The construct tree illustrated in the previous section is only a depiction of what a human readable(ish) view might look like 2. **Constructs** 1. ***Nodes*** - CDK application constructs at the root and drill down to the constructs it contains (1+ cloud resources.) Initially, we will focus our attention on AWS resources, but the model is extensible to cloud components from any provider - 2. ***Metadata -*** Constructs will expose metadata which will include an array of objects associated with the construct. + 2. ***Metadata -*** Constructs will expose metadata which will include an array of objects associated with the construct. 1. ***Properties*** - Higher level constructs (L2 and L3) will expose properties and values of the resources they contain. This will especially be useful for constructs that are opinionated and set defaults on behalf of developers 2. ***Errors/Warnings*** - Errors and warnings that are produced during synthesis indicating validation failures, deprecation notices, guidance, etc will be included in the tree. These are emitted at the construct level although the message may point towards a specific property. 3. **Assets** - Assets information will be included in the construct tree. Assets represent actions that the CDK *will* take ahead of stack deployments (i.e. S3 assets are zipped and uploaded directly, Docker images are uploaded to ECR). The asset metadata in the tree is the S3 key that an asset *would* have @@ -173,7 +172,6 @@ As a temporary placeholder, here's the implementation of the prototype construct }] }] }, - ... ``` ### Construct metadata @@ -212,7 +210,7 @@ export interface IDisplayable { ## Future Possibilities -We do not expect customers to have to interact directly with the outputs that the CDK produces as a part of the cloud assembly. Publishing the CDK construct tree as an output to the cloud assembly is intended to be a starting point to build on and create experiences based on it. +We do not expect customers to have to interact directly with the outputs that the CDK produces as a part of the cloud assembly. Publishing the CDK construct tree as an output to the cloud assembly is intended to be a starting point to build on and create experiences based on it. ### Tooling From 9ae5abc6f1d4ae6e6444edb59f46e4cff411ca1b Mon Sep 17 00:00:00 2001 From: Shiv Lakshminarayan Date: Thu, 12 Sep 2019 07:27:21 -0700 Subject: [PATCH 3/9] getting rid of placeholder construct tree map as it's a little confusing --- design/construct-tree.md | 31 ------------------------------- 1 file changed, 31 deletions(-) diff --git a/design/construct-tree.md b/design/construct-tree.md index 3c77fb5e67798..779c596a54722 100644 --- a/design/construct-tree.md +++ b/design/construct-tree.md @@ -143,37 +143,6 @@ The following metadata properties will be included by the construct that produce **TODO** - add concrete walkthrough example of a CDK application and the construct-tree flat map it would produce -As a temporary placeholder, here's the implementation of the prototype construct referenced in the overview. It's a representation of the construct-tree of the [CDK Workshop](https://cdkworkshop.com/) application. - -```json -{ - "id": "App", - "path": "", - "children": [{ - "id": "CdkWorkshopStack", - "path": "CdkWorkshopStack", - "children": [{ - "id": "HelloHandler", - "path": "CdkWorkshopStack/HelloHandler", - "children": [{ - "id": "ServiceRole", - "path": "CdkWorkshopStack/HelloHandler/ServiceRole", - "children": [{ - "id": "Resource", - "path": "CdkWorkshopStack/HelloHandler/ServiceRole/Resource", - "metadata": { - "resourceType": "AWS::IAM::Role", - "logicalId": "HelloHandlerServiceRole11EF7C63" - }, - "links": [{ - "sourcePath": "CdkWorkshopStack/HelloHandler/Resource", - "targetPath": "CdkWorkshopStack/HelloHandler/ServiceRole/Resource", - "attribute": "Arn" - }] - }] - }, -``` - ### Construct metadata We will also introduce an interface that constructs *can* implement to provide metadata such as properties, additional context, anything a construct would want to report back into the rendered view. Constructs will be able to supply information about themselves for inclusion into the context tree. From 0b625a49a838ff5d827de0d75dbf6462ffd5374f Mon Sep 17 00:00:00 2001 From: Shiv Lakshminarayan Date: Thu, 12 Sep 2019 17:21:26 -0700 Subject: [PATCH 4/9] update properties, included a placeholder sample of the schema --- design/construct-tree.md | 219 +++++++++++++++++++++++++++++++++------ 1 file changed, 186 insertions(+), 33 deletions(-) diff --git a/design/construct-tree.md b/design/construct-tree.md index 779c596a54722..2a4a4d43f4c03 100644 --- a/design/construct-tree.md +++ b/design/construct-tree.md @@ -1,4 +1,4 @@ -## RFC: PUBLISH CDK CONSTRUCT TREE +## RFC: PUBLISH CDK CONSTRUCT TREE ## Overview @@ -24,7 +24,7 @@ Developers author their CDK applications by leveraging higher-level intent based As an example, the following CDK application for an Application Load Balancer includes constructs from Auto Scaling, VPC, and Application Load Balancing. -```ts +```typescript const vpc = new ec2.Vpc(this, 'VPC'); const asg = new autoscaling.AutoScalingGroup(this, 'ASG', { @@ -61,11 +61,10 @@ Continuing with this example, inspecting the the cloud assembly’s CloudFormati In this scenario, these are some details that are not intuitive without diving deeper: -> What resources does the AutoScalingGroup construct contain? the VPC construct? +>1. What resources does the AutoScalingGroup construct contain? the VPC construct? etc. +>2. What properties were the resources initialized with? Can they be changed? +>3. Is the relationship between these constructs captured in the cloud assembly correctly? -> What properties were the resources initialized with? Can they be changed? - -> Is the relationship between these constructs captured in the cloud assembly correctly? When deployments fail, or provisioned resources don’t hold the properties that one might expect, developers can’t figure out what properties that resources such as `NatGateway, AutoScalingGroup, Subnets, Route Tables` were configured with without dropping into the generated CloudFormation template. It can take a good amount of clicking and digging before developers can figure out what happened and almost certainly needs them to get away from their favourite IDE and switch context to find out. @@ -78,7 +77,7 @@ Let’s take another look at an example of what a construct tree might look like This is a Lambda function that has it’s handler code written as an asset: -```ts +```typescript new lambda.Function(this, 'HelloWorldHandler', { runtime: lambda.Runtime.NODE_JS_8_10, code: lambda.Code.directory('lambda'), @@ -104,15 +103,17 @@ This list describes only the minimal set of requirements from this feature. Afte 1. **Format** - Publish underlying data as a `.json` file that contains the information required to render a construct tree view. The construct tree illustrated in the previous section is only a depiction of what a human readable(ish) view might look like 2. **Constructs** 1. ***Nodes*** - CDK application constructs at the root and drill down to the constructs it contains (1+ cloud resources.) Initially, we will focus our attention on AWS resources, but the model is extensible to cloud components from any provider - 2. ***Metadata -*** Constructs will expose metadata which will include an array of objects associated with the construct. - 1. ***Properties*** - Higher level constructs (L2 and L3) will expose properties and values of the resources they contain. This will especially be useful for constructs that are opinionated and set defaults on behalf of developers - 2. ***Errors/Warnings*** - Errors and warnings that are produced during synthesis indicating validation failures, deprecation notices, guidance, etc will be included in the tree. These are emitted at the construct level although the message may point towards a specific property. - 3. **Assets** - Assets information will be included in the construct tree. Assets represent actions that the CDK *will* take ahead of stack deployments (i.e. S3 assets are zipped and uploaded directly, Docker images are uploaded to ECR). The asset metadata in the tree is the S3 key that an asset *would* have + 2. **Types/Hierarchy** - Types and hierarchy information will contain the details around the level of abstraction (L1, L2, L3..), service(s), and resource(s) that they represent. + 3. ***Metadata -*** Constructs will expose metadata which will include an array of objects associated with the construct. + 1. **Location** - The full path of the location where the construct is defined. This will support use cases where the construct-tree nodes can support navigability back to the declaration in the IDE itself. + 2. ***Properties*** - Constructs will opt into the set of properties, and these will be exposed. Higher level constructs (L2 and L3) will expose properties and values of the resources they contain and not necessarily enumerate every property that’s available for a resource. This will especially be useful for constructs that are opinionated and set defaults on behalf of developers. Low level constructs (L1) will expose all properties that are configured. + 3. ***Errors/Warnings*** - Errors and warnings that are produced during synthesis indicating validation failures, deprecation notices, guidance, etc will be included in the tree. These are emitted at the construct level although the message may point towards a specific property. The construct tree should be complete and include validation failures for all resources in the CDK application rather than just just fail on the first error. + 4. **Assets** - Assets information will be included in the construct tree. Assets represent actions that the CDK *will* take ahead of stack deployments (i.e. S3 assets are zipped and uploaded directly, Docker images are uploaded to ECR). The asset metadata in the tree is the S3 key that an asset *would* have 3. **Local** - The construct tree is produced at synthesis time. Connectivity to AWS and CloudFormation should not be required to produce a tree view of a CDK application. ## Approach -At a high-level, we will render the `construct tree` through a construct tree data model and construct metadata interface that constructs will implement: +At a high-level, we will render the `construct tree` through a construct tree data model and construct metadata interface that constructs will implement. This section will describe the components of the JSON structure that will represent the construct-tree: ### Construct Tree Data Model @@ -122,26 +123,178 @@ The construct tree will be a list of paths that are indexed into a flat map of c ### Construct properties -|Property |Type |Required |Description | -|--- |--- |--- |--- | -|id |string |Required |id of the construct within the current scope | -|path |string |Required |full, absolute path of the construct within the tree | +|Property |Type |Required |Description | +|--- |--- |--- |--- | +|id |string |Required |id of the construct within the current scope | +|path |string |Required |full, absolute path of the construct within the tree | +|children |Array |Not Required |All direct children of this construct. Array of the absolute paths of the constructs. Will be used to walk entire list of constructs | +|metadata |Array |Not Required |Metadata describing all constructs/resources that are encapsulated by the construct | ### Metadata Properties -The following metadata properties will be included by the construct that produces the `tree.json` output. - -|Property |Type |Required |Description | -|--- |--- |--- |--- | -|sourceLocation |sourceLocation |Required |location in source code where the construct is defined | -|url |String |Required |array of metadata objects associated with this construct. | -|renderedView |Array |Not Required |constructs can fill in arbitrary. ClouFormation agnostic. CFN constructs could pre-populate it with the output of `this.toCloudFormation()` | -|children |Array |Not Required |All direct children of this construct. Array of the absolute paths of the constructs. Will be used to walk entire list of constructs | -|errors |Array |Not Required |array of errors associated with this construct. | -|warnings |Array |Not Required |array of warnings associated with this construct | -|description |string |Not Required |description of the construct | - -**TODO** - add concrete walkthrough example of a CDK application and the construct-tree flat map it would produce +The following metadata properties will be included by the construct that produces the `tree.json` output. + +|Property |Type |Required |Description | +|--- |--- |--- |--- | +|sourceLocation |string |Required |location in source code where the construct is defined | +|resourceType |string |Required |type of resource (schema / classification terminology TBD) | +|description |string |Not Required |description of the construct | +|renderedView |Array |Not Required |constructs can fill in arbitrary. ClouFormation agnostic. CFN constructs could pre-populate it with the output of `this.toCloudFormation()` | +|errors |Array |Not Required |array of errors associated with this construct. | +|warnings |Array |Not Required |array of warnings associated with this construct | + +This is a temporary placeholder (WIP) of the construct tree in the JSON schema format described above. It’s a representation of the tree for the prototype referenced earlier in the Overview section of the CDK app in the [cdkworkshop](https://cdkworkshop.com/) + +**TODO** - add detail and a more concrete walkthrough with samples of all the attributes referenced in construct and metadata properties. + +```json +{ + "id": "App", + "path": "", + "metadata": { + "sourceLocation": "", + "resourceType": "AWS::CDK::App" + }, + "children": [ + { + "id": "CdkWorkshopStack", + "path": "CdkWorkshopStack", + "resourceType": "AWS::CloudFormation::Stack", + "metadata": [ + { + "sourceLocation": "/Users/shivlaks/Documents/CDK/projects/august/typescript/1.6.1/workshop/lib/cdkworkshop-stack.ts:8:22", + "description": "Construct for a CloudFormation Stack", + "renderedView": [ + { + "notificationArn": "sample-notification-arn", + "retentionPolicy": "RETAIN", + "role-arn": "sample-role-arn" + } + ], + "warnings": [ + { + "deprecatedParameter": "`transform` is deprecated, use `transforms` instead" + } + ] + } + ], + "children": [ + { + "id": "HelloHandler", + "path": "CdkWorkshopStack/HelloHandler", + "resourceType": "AWS::Lambda::Function", + "metadata": [ + { + "sourceLocation": "sample-source-location", + "renderedView": [ + { + "memorySize": "128MB", + "Role": "sample-role", + "Tags": [ + { + "sample-tag": "yay-cdk", + "safe-to-delete": "yup" + } + ] + } + ] + } + ], + "children": [ + { + "id": "ServiceRole", + "path": "CdkWorkshopStack/HelloHandler/ServiceRole", + "resourceType": "AWS::IAM::Role", + "metadata": [ + { + "sourceLocation": "sample-source-location", + "renderedView": "renderedView", + "errrors": [], + "warnings": [] + }, + { + "id": "Resource", + "path": "CdkWorkshopStack/HelloHandler/Resource", + "resourceType": "AWS::Lambda::Function", + "metadata": [ + { + "sourceLocation": "sample-source-location", + "renderedView": [ + { + "logicalId": "HelloHandler2E4FBA4D" + } + ] + } + ] + }, + { + "id": "Code", + "path": "CdkWorkshopStack/HelloHandler/Code", + "resourceType": "AWS::CDK::Asset", + "metadata": [ + { + "sourceLocation": "sample-source-location" + } + ], + "children": [ + { + "id": "S3Bucket", + "path": "CdkWorkshopStack/HelloHandler/Code/S3Bucket", + "resourecType": "AWS::S3::Bucket", + "metadata": [ + { + "sourceLocation": "sample-source-location" + } + ] + }, + { + "id": "S3VersionKey", + "path": "CdkWorkshopStack/HelloHandler/Code/S3VersionKey", + "resourceType": "AWS::S3::Attribute", + "metadata": [ + { + "sourceLocation": "sample-source-location" + } + ] + }, + { + "id": "AssetBucket", + "path": "CdkWorkshopStack/HelloHandler/Code/AssetBucket", + "resourceType": "AWS::S3::Bucket", + "metadata": [ + { + "sourceLocation": "sample-source-location" + } + ] + } + ] + } + ], + "children": [ + { + "id": "Resource", + "path": "CdkWorkshopStack/HelloHandler/ServiceRole/Resource", + "resourceType": "AWS::IAM::Role", + "metadata": [ + { + "sourceLocation": "sample-source-location", + "renderedView": [ + { + "logicalId": "HelloHandlerServiceRole11EF7C63" + } + ] + } + ] + } + ] + } + ] + } + ] + } + ] +} +``` ### Construct metadata @@ -149,7 +302,7 @@ We will also introduce an interface that constructs *can* implement to provide m The context tree provides the skeleton and the structure for rendering a tree-view of a CDK application. The proposed Interface will be added to `construct.ts` in the core library and implemented by `constructs` that have additional information to contribute to the construct tree : -```ts +```typescript /*** * Displayable information that a construct will contribute towards the context tree* * such as dependencies, properties, links, documentation, etc @@ -159,8 +312,8 @@ export interface IDisplayable { // id of the construct within the current scope* readonly id: string - //array of metadata objects associated with this construct.* - readonly url: string + // type of the resource (could be a CDK, CloudFormation, third-party, etc)* + readonly resourceType: string // Other displayable metadata* readonly metadata: [key: string]: string; @@ -187,4 +340,4 @@ With a construct tree model in place and construct tree being published as a `.j **Diff** diff the construct tree against the resources that are deployed in the cloud. This capability would require cloud capability but we should always be able to render a construct tree without the need to connect. -**Navigability** include the location in code where the construct is defined. This would start to pave the way towards building tooling and extensions that IDE's can leverage. \ No newline at end of file +**Navigability** include the location in code where the construct is defined. This would start to pave the way towards building tooling and extensions that IDE's can leverage. From 88f173f6a47cbb6604026427b885fe099d24932b Mon Sep 17 00:00:00 2001 From: Shiv Lakshminarayan Date: Mon, 30 Sep 2019 21:24:38 -0700 Subject: [PATCH 5/9] clean up formatting to address feedback --- design/construct-tree.md | 103 ++++++++++++++++----------------------- 1 file changed, 42 insertions(+), 61 deletions(-) diff --git a/design/construct-tree.md b/design/construct-tree.md index 2a4a4d43f4c03..333a1c3b70b9f 100644 --- a/design/construct-tree.md +++ b/design/construct-tree.md @@ -4,7 +4,7 @@ This is a strategy for publishing the construct tree which represents a CDK app as a part of the Cloud Assembly that the CDK generates. -***Goal*: Expose the** **`construct tree`** **for CDK Apps as an artifact of the Cloud Assembly** +**Goal: Expose the `construct tree` for CDK Apps as an artifact of the Cloud Assembly** This proposal details the motivation behind the goal, requirements, specification of the construct tree, and the proposed design. The scope of the design is focused on defining what the construct tree should include and the approach to produce it as an output. @@ -25,36 +25,35 @@ Developers author their CDK applications by leveraging higher-level intent based As an example, the following CDK application for an Application Load Balancer includes constructs from Auto Scaling, VPC, and Application Load Balancing. ```typescript - const vpc = new ec2.Vpc(this, 'VPC'); +const vpc = new ec2.Vpc(this, 'VPC'); - const asg = new autoscaling.AutoScalingGroup(this, 'ASG', { - vpc, - instanceType: ec2.InstanceType.of(ec2.InstanceClass.T2, ec2.InstanceSize.MICRO), - machineImage: new ec2.AmazonLinuxImage(), - }); +const asg = new autoscaling.AutoScalingGroup(this, 'ASG', { + vpc, + instanceType: ec2.InstanceType.of(ec2.InstanceClass.T2, ec2.InstanceSize.MICRO), + machineImage: new ec2.AmazonLinuxImage(), +}); - const lb = new elbv2.ApplicationLoadBalancer(this, 'LB', { - vpc, - internetFacing: true - }); +const lb = new elbv2.ApplicationLoadBalancer(this, 'LB', { + vpc, + internetFacing: true +}); - const listener = lb.addListener('Listener', { - port: 80, - }); +const listener = lb.addListener('Listener', { + port: 80, +}); - listener.addTargets('Target', { - port: 80, - targets: [asg] - }); +listener.addTargets('Target', { + port: 80, + targets: [asg] +}); - listener.connections.allowDefaultPortFromAnyIpv4('Open to the world'); +listener.connections.allowDefaultPortFromAnyIpv4('Open to the world'); - asg.scaleOnRequestCount('AModestLoad', { - targetRequestsPerSecond: 1 - }); +asg.scaleOnRequestCount('AModestLoad', { + targetRequestsPerSecond: 1 +}); ``` - Running `cdk synth` on this application produces ~800 lines of CloudFormation template and represents over 35 AWS resources spanning 5 services. The generated template is part of the cloud assembly and will be used subsequently by deploy and is staged in the `cdk.out` directory. This showcases the power of the CDK in the ability to define high level abstractions of infrastructure in familiar languages that can be reused. Continuing with this example, inspecting the the cloud assembly’s CloudFormation template doesn’t help gain any insight when behavior does not match expectations. @@ -117,33 +116,30 @@ At a high-level, we will render the `construct tree` through a construct tree da ### Construct Tree Data Model -A feature will be added to the CDK core module so that any CDK application will produce an artifact called `tree.json` during calls to `cdk synth` and include it as an output to the cloud assembly in `cdk.out`. Developers will be able to opt-out if this functionality is not desired or needed through a flag in the CLI or set it as a preference. +A feature will be added to the CDK core module so that any CDK application will produce an artifact called `tree.json` during calls to `cdk synth` and include it as an output to the cloud assembly in `cdk.out`. -The construct tree will be a list of paths that are indexed into a flat map of constructs. The file structure itself represents the tree. This information will need to include the following to start rendering a flat map of constructs: +The construct tree will be a list of paths that are indexed into a map of constructs. The file structure itself represents the tree. This information will need to include the following to start rendering a map of constructs: ### Construct properties -|Property |Type |Required |Description | -|--- |--- |--- |--- | -|id |string |Required |id of the construct within the current scope | -|path |string |Required |full, absolute path of the construct within the tree | -|children |Array |Not Required |All direct children of this construct. Array of the absolute paths of the constructs. Will be used to walk entire list of constructs | -|metadata |Array |Not Required |Metadata describing all constructs/resources that are encapsulated by the construct | +|Property |Type |Required |Description | +|--- |--- |--- |--- | +|id |string |Required |id of the construct within the current scope | +|path |string |Required |full, absolute path of the construct within the tree | +|children |Array |Not Required |All direct children of this construct. Array of the absolute paths of the constructs. Will be used to walk entire list of constructs | +|metadata |Array |Not Required |Metadata describing all constructs/resources that are encapsulated by the construct | ### Metadata Properties The following metadata properties will be included by the construct that produces the `tree.json` output. -|Property |Type |Required |Description | -|--- |--- |--- |--- | -|sourceLocation |string |Required |location in source code where the construct is defined | -|resourceType |string |Required |type of resource (schema / classification terminology TBD) | -|description |string |Not Required |description of the construct | -|renderedView |Array |Not Required |constructs can fill in arbitrary. ClouFormation agnostic. CFN constructs could pre-populate it with the output of `this.toCloudFormation()` | -|errors |Array |Not Required |array of errors associated with this construct. | -|warnings |Array |Not Required |array of warnings associated with this construct | - -This is a temporary placeholder (WIP) of the construct tree in the JSON schema format described above. It’s a representation of the tree for the prototype referenced earlier in the Overview section of the CDK app in the [cdkworkshop](https://cdkworkshop.com/) +|Property |Type |Required |Description | +|--- |--- |--- |--- | +|sourceLocation |string |Required |location in source code where the construct is defined | +|description |string |Not Required |description of the construct | +|renderedView |Array |Not Required |constructs can fill in arbitrary metadata such as configuration. ClouFormation agnostic. CFN constructs could pre-populate it with the output of `this.toCloudFormation()` | +|errors |Array |Not Required |array of errors associated with this construct. | +|warnings |Array |Not Required |array of warnings associated with this construct | **TODO** - add detail and a more concrete walkthrough with samples of all the attributes referenced in construct and metadata properties. @@ -153,16 +149,14 @@ This is a temporary placeholder (WIP) of the construct tree in the JSON schema f "path": "", "metadata": { "sourceLocation": "", - "resourceType": "AWS::CDK::App" }, "children": [ { "id": "CdkWorkshopStack", "path": "CdkWorkshopStack", - "resourceType": "AWS::CloudFormation::Stack", "metadata": [ { - "sourceLocation": "/Users/shivlaks/Documents/CDK/projects/august/typescript/1.6.1/workshop/lib/cdkworkshop-stack.ts:8:22", + "sourceLocation": "/lib/cdkworkshop-stack.ts:8:22", "description": "Construct for a CloudFormation Stack", "renderedView": [ { @@ -182,7 +176,6 @@ This is a temporary placeholder (WIP) of the construct tree in the JSON schema f { "id": "HelloHandler", "path": "CdkWorkshopStack/HelloHandler", - "resourceType": "AWS::Lambda::Function", "metadata": [ { "sourceLocation": "sample-source-location", @@ -204,18 +197,14 @@ This is a temporary placeholder (WIP) of the construct tree in the JSON schema f { "id": "ServiceRole", "path": "CdkWorkshopStack/HelloHandler/ServiceRole", - "resourceType": "AWS::IAM::Role", "metadata": [ { "sourceLocation": "sample-source-location", "renderedView": "renderedView", - "errrors": [], - "warnings": [] }, { "id": "Resource", "path": "CdkWorkshopStack/HelloHandler/Resource", - "resourceType": "AWS::Lambda::Function", "metadata": [ { "sourceLocation": "sample-source-location", @@ -230,7 +219,6 @@ This is a temporary placeholder (WIP) of the construct tree in the JSON schema f { "id": "Code", "path": "CdkWorkshopStack/HelloHandler/Code", - "resourceType": "AWS::CDK::Asset", "metadata": [ { "sourceLocation": "sample-source-location" @@ -250,7 +238,6 @@ This is a temporary placeholder (WIP) of the construct tree in the JSON schema f { "id": "S3VersionKey", "path": "CdkWorkshopStack/HelloHandler/Code/S3VersionKey", - "resourceType": "AWS::S3::Attribute", "metadata": [ { "sourceLocation": "sample-source-location" @@ -260,7 +247,6 @@ This is a temporary placeholder (WIP) of the construct tree in the JSON schema f { "id": "AssetBucket", "path": "CdkWorkshopStack/HelloHandler/Code/AssetBucket", - "resourceType": "AWS::S3::Bucket", "metadata": [ { "sourceLocation": "sample-source-location" @@ -274,7 +260,6 @@ This is a temporary placeholder (WIP) of the construct tree in the JSON schema f { "id": "Resource", "path": "CdkWorkshopStack/HelloHandler/ServiceRole/Resource", - "resourceType": "AWS::IAM::Role", "metadata": [ { "sourceLocation": "sample-source-location", @@ -309,14 +294,11 @@ The context tree provides the skeleton and the structure for rendering a tree-vi */ export interface IDisplayable { - // id of the construct within the current scope* - readonly id: string - - // type of the resource (could be a CDK, CloudFormation, third-party, etc)* - readonly resourceType: string + // id of the construct within the current scope + readonly displayName: string - // Other displayable metadata* - readonly metadata: [key: string]: string; + // Other displayable attributes + readonly attributes: [key: string]: string; /* method that supports extensibility on what can be displayed without * adding more methods/properties on IDisplayable @@ -328,7 +310,6 @@ export interface IDisplayable { ## Unresolved Questions * Experience across all supported languages - nuances, considerations, etc -* Performance - `synth` can take longer to render with this added construct and additional ## Future Possibilities From 3e55a1e2b177d617444d305338a80d82d51364a4 Mon Sep 17 00:00:00 2001 From: Shiv Lakshminarayan Date: Sun, 13 Oct 2019 01:27:40 -0700 Subject: [PATCH 6/9] add experimental header, rename renderedView -> properties, added placeholders to describe source of construct properties --- design/construct-tree.md | 289 ++++++++++++++++++++------------------- 1 file changed, 148 insertions(+), 141 deletions(-) diff --git a/design/construct-tree.md b/design/construct-tree.md index 333a1c3b70b9f..16f7ac8ad5929 100644 --- a/design/construct-tree.md +++ b/design/construct-tree.md @@ -122,170 +122,177 @@ The construct tree will be a list of paths that are indexed into a map of constr ### Construct properties -|Property |Type |Required |Description | -|--- |--- |--- |--- | -|id |string |Required |id of the construct within the current scope | -|path |string |Required |full, absolute path of the construct within the tree | -|children |Array |Not Required |All direct children of this construct. Array of the absolute paths of the constructs. Will be used to walk entire list of constructs | -|metadata |Array |Not Required |Metadata describing all constructs/resources that are encapsulated by the construct | +|Property |Type |Required |Source | Description | +|--- |--- |--- |--- | --- | +|id |string |Required |TODO | id of the construct within the current scope | +|path |string |Required |TODO | full, absolute path of the construct within the tree | +|children |Array |Not Required |TODO | All direct children of this construct. Array of the absolute paths of the constructs. Will be used to walk entire list of constructs | +|metadata |Array |Not Required |TODO | Metadata describing all constructs/resources that are encapsulated by the construct | ### Metadata Properties The following metadata properties will be included by the construct that produces the `tree.json` output. -|Property |Type |Required |Description | -|--- |--- |--- |--- | -|sourceLocation |string |Required |location in source code where the construct is defined | -|description |string |Not Required |description of the construct | -|renderedView |Array |Not Required |constructs can fill in arbitrary metadata such as configuration. ClouFormation agnostic. CFN constructs could pre-populate it with the output of `this.toCloudFormation()` | -|errors |Array |Not Required |array of errors associated with this construct. | -|warnings |Array |Not Required |array of warnings associated with this construct | +|Property |Type |Required |Source | Description | +|--- |--- |--- |--- | --- | +|sourceLocation |string |Required |TODO | location in source code where the construct is defined | +|description |string |Not Required |TODO | description of the construct | +|properties |Array |Not Required |TODO | constructs can fill in arbitrary metadata such as configuration. ClouFormation agnostic. CFN constructs could pre-populate it with the output of `this.toCloudFormation()` | +|errors |Array |Not Required |TODO | array of errors associated with this construct. | +|warnings |Array |Not Required |TODO | array of warnings associated with this construct | **TODO** - add detail and a more concrete walkthrough with samples of all the attributes referenced in construct and metadata properties. ```json { - "id": "App", - "path": "", - "metadata": { - "sourceLocation": "", - }, - "children": [ - { - "id": "CdkWorkshopStack", - "path": "CdkWorkshopStack", - "metadata": [ - { - "sourceLocation": "/lib/cdkworkshop-stack.ts:8:22", - "description": "Construct for a CloudFormation Stack", - "renderedView": [ + "version": "1.0-experimental", + "tree": { + "id": "App", + "path": "/", + "metadata": { + "sourceLocation": "" + }, + "children": [ { - "notificationArn": "sample-notification-arn", - "retentionPolicy": "RETAIN", - "role-arn": "sample-role-arn" - } - ], - "warnings": [ - { - "deprecatedParameter": "`transform` is deprecated, use `transforms` instead" - } - ] - } - ], - "children": [ - { - "id": "HelloHandler", - "path": "CdkWorkshopStack/HelloHandler", - "metadata": [ - { - "sourceLocation": "sample-source-location", - "renderedView": [ - { - "memorySize": "128MB", - "Role": "sample-role", - "Tags": [ - { - "sample-tag": "yay-cdk", - "safe-to-delete": "yup" - } - ] - } - ] - } - ], - "children": [ - { - "id": "ServiceRole", - "path": "CdkWorkshopStack/HelloHandler/ServiceRole", - "metadata": [ - { - "sourceLocation": "sample-source-location", - "renderedView": "renderedView", - }, - { - "id": "Resource", - "path": "CdkWorkshopStack/HelloHandler/Resource", - "metadata": [ - { - "sourceLocation": "sample-source-location", - "renderedView": [ - { - "logicalId": "HelloHandler2E4FBA4D" - } - ] - } - ] - }, - { - "id": "Code", - "path": "CdkWorkshopStack/HelloHandler/Code", - "metadata": [ - { - "sourceLocation": "sample-source-location" - } - ], - "children": [ - { - "id": "S3Bucket", - "path": "CdkWorkshopStack/HelloHandler/Code/S3Bucket", - "resourecType": "AWS::S3::Bucket", - "metadata": [ - { - "sourceLocation": "sample-source-location" - } - ] - }, - { - "id": "S3VersionKey", - "path": "CdkWorkshopStack/HelloHandler/Code/S3VersionKey", - "metadata": [ - { - "sourceLocation": "sample-source-location" - } - ] - }, + "id": "CdkWorkshopStack", + "path": "CdkWorkshopStack", + "metadata": [ { - "id": "AssetBucket", - "path": "CdkWorkshopStack/HelloHandler/Code/AssetBucket", - "metadata": [ - { - "sourceLocation": "sample-source-location" - } - ] + "sourceLocation": "/lib/cdkworkshop-stack.ts:8:22", + "description": "Construct for a CloudFormation Stack", + "properties": [ + { + "notificationArn": "sample-notification-arn", + "retentionPolicy": "RETAIN", + "role-arn": "sample-role-arn" + } + ], + "warnings": [ + { + "deprecatedParameter": "`transform` is deprecated, use `transforms` instead" + } + ] } - ] - } - ], - "children": [ - { - "id": "Resource", - "path": "CdkWorkshopStack/HelloHandler/ServiceRole/Resource", - "metadata": [ + ], + "children": [ { - "sourceLocation": "sample-source-location", - "renderedView": [ - { - "logicalId": "HelloHandlerServiceRole11EF7C63" - } - ] + "id": "HelloHandler", + "path": "CdkWorkshopStack/HelloHandler", + "metadata": [ + { + "sourceLocation": "sample-source-location", + "properties": [ + { + "memorySize": "128MB", + "Role": "sample-role", + "Tags": [ + { + "sample-tag": "yay-cdk", + "safe-to-delete": "yup" + } + ] + } + ] + } + ], + "children": [ + { + "id": "ServiceRole", + "path": "CdkWorkshopStack/HelloHandler/ServiceRole", + "metadata": [ + { + "sourceLocation": "sample-source-location", + "properties": "...properties" + }, + { + "id": "Resource", + "path": "CdkWorkshopStack/HelloHandler/Resource", + "metadata": [ + { + "sourceLocation": "sample-source-location", + "properties": [ + { + "logicalId": "HelloHandler2E4FBA4D" + } + ] + } + ] + }, + { + "id": "Code", + "path": "CdkWorkshopStack/HelloHandler/Code", + "metadata": [ + { + "sourceLocation": "sample-source-location" + } + ], + "children": [ + { + "id": "S3Bucket", + "path": "CdkWorkshopStack/HelloHandler/Code/S3Bucket", + "resourecType": "AWS::S3::Bucket", + "metadata": [ + { + "sourceLocation": "sample-source-location" + } + ] + }, + { + "id": "S3VersionKey", + "path": "CdkWorkshopStack/HelloHandler/Code/S3VersionKey", + "metadata": [ + { + "sourceLocation": "sample-source-location" + } + ] + }, + { + "id": "AssetBucket", + "path": "CdkWorkshopStack/HelloHandler/Code/AssetBucket", + "metadata": [ + { + "sourceLocation": "sample-source-location" + } + ] + } + ] + } + ], + "children": [ + { + "id": "Resource", + "path": "CdkWorkshopStack/HelloHandler/ServiceRole/Resource", + "metadata": [ + { + "sourceLocation": "sample-source-location", + "properties": [ + { + "logicalId": "HelloHandlerServiceRole11EF7C63" + } + ] + } + ] + } + ] + } + ] } - ] - } - ] + ] } - ] - } - ] + ] } - ] } ``` ### Construct metadata -We will also introduce an interface that constructs *can* implement to provide metadata such as properties, additional context, anything a construct would want to report back into the rendered view. Constructs will be able to supply information about themselves for inclusion into the context tree. +We will also introduce an interface that constructs *can* implement to provide metadata such as properties, +additional context, anything a construct would want to report back as properties. +Constructs will be able to supply information about themselves for inclusion into the context tree. -The context tree provides the skeleton and the structure for rendering a tree-view of a CDK application. The proposed Interface will be added to `construct.ts` in the core library and implemented by `constructs` that have additional information to contribute to the construct tree : +The context tree provides the skeleton and the structure for rendering a tree-view of a CDK application. +The proposed Interface will be added to `construct.ts` in the core library and implemented by `constructs` +that have additional information to contribute to the construct tree : ```typescript /*** From 46a72223a5267322c8fb87965470526b985b99c6 Mon Sep 17 00:00:00 2001 From: Shiv Lakshminarayan Date: Thu, 17 Oct 2019 03:20:27 -0700 Subject: [PATCH 7/9] updates to tree, removed id as a nested attribute, created placeholders to fill in source of metadata --- design/construct-tree.md | 253 +++++++++++++++++++-------------------- 1 file changed, 120 insertions(+), 133 deletions(-) diff --git a/design/construct-tree.md b/design/construct-tree.md index 16f7ac8ad5929..6af6e1f859abc 100644 --- a/design/construct-tree.md +++ b/design/construct-tree.md @@ -124,7 +124,6 @@ The construct tree will be a list of paths that are indexed into a map of constr |Property |Type |Required |Source | Description | |--- |--- |--- |--- | --- | -|id |string |Required |TODO | id of the construct within the current scope | |path |string |Required |TODO | full, absolute path of the construct within the tree | |children |Array |Not Required |TODO | All direct children of this construct. Array of the absolute paths of the constructs. Will be used to walk entire list of constructs | |metadata |Array |Not Required |TODO | Metadata describing all constructs/resources that are encapsulated by the construct | @@ -137,156 +136,140 @@ The following metadata properties will be included by the construct that produce |--- |--- |--- |--- | --- | |sourceLocation |string |Required |TODO | location in source code where the construct is defined | |description |string |Not Required |TODO | description of the construct | -|properties |Array |Not Required |TODO | constructs can fill in arbitrary metadata such as configuration. ClouFormation agnostic. CFN constructs could pre-populate it with the output of `this.toCloudFormation()` | -|errors |Array |Not Required |TODO | array of errors associated with this construct. | -|warnings |Array |Not Required |TODO | array of warnings associated with this construct | +|properties |Array |Not Required |TODO | constructs can fill in arbitrary metadata such as configuration. CloudFormation agnostic. CFN constructs will include the properties in `this.toCloudFormation()` | **TODO** - add detail and a more concrete walkthrough with samples of all the attributes referenced in construct and metadata properties. ```json { - "version": "1.0-experimental", - "tree": { - "id": "App", - "path": "/", - "metadata": { - "sourceLocation": "" - }, - "children": [ + "version": "1.0-experimental", + "tree": { + "App": { + "path": "/", + "metadata": { + "sourceLocation": "" + }, + "children": { + "CdkWorkshopStack": { + "path": "CdkWorkshopStack", + "metadata": [ { - "id": "CdkWorkshopStack", - "path": "CdkWorkshopStack", - "metadata": [ + "sourceLocation": "/lib/cdkworkshop-stack.ts:8:22", + "description": "Construct for a CloudFormation Stack", + "properties": { + "notificationArn": "sample-notification-arn", + "retentionPolicy": "RETAIN", + "role-arn": "sample-role-arn" + }, + "warnings": [ + { + "deprecatedParameter": "`transform` is deprecated, use `transforms` instead" + } + ] + } + ], + "children": { + "HelloHandler": { + "path": "CdkWorkshopStack/HelloHandler", + "metadata": [ + { + "sourceLocation": "sample-source-location", + "properties": { + "memorySize": "128MB", + "Role": "sample-role", + "Tags": [ + { + "sample-tag": "yay-cdk", + "safe-to-delete": "yup" + } + ] + } + } + ], + "children": { + "ServiceRole": { + "path": "CdkWorkshopStack/HelloHandler/ServiceRole", + "metadata": [ { - "sourceLocation": "/lib/cdkworkshop-stack.ts:8:22", - "description": "Construct for a CloudFormation Stack", - "properties": [ - { - "notificationArn": "sample-notification-arn", - "retentionPolicy": "RETAIN", - "role-arn": "sample-role-arn" - } - ], - "warnings": [ - { - "deprecatedParameter": "`transform` is deprecated, use `transforms` instead" - } - ] + "sourceLocation": "sample-source-location", + "properties": { + "something-cool": "about-this-role" + } } - ], - "children": [ + ] + }, + "Resource": { + "path": "CdkWorkshopStack/HelloHandler/Resource", + "metadata": [ { - "id": "HelloHandler", - "path": "CdkWorkshopStack/HelloHandler", - "metadata": [ - { - "sourceLocation": "sample-source-location", - "properties": [ - { - "memorySize": "128MB", - "Role": "sample-role", - "Tags": [ - { - "sample-tag": "yay-cdk", - "safe-to-delete": "yup" - } - ] - } - ] - } - ], - "children": [ + "sourceLocation": "sample-source-location", + "properties": { + "logicalId": "HelloHandler2E4FBA4D" + } + } + ] + }, + "Code": { + "path": "CdkWorkshopStack/HelloHandler/Code", + "metadata": [ + { + "sourceLocation": "sample-source-location" + } + ], + "children": { + "S3Bucket": { + "path": "CdkWorkshopStack/HelloHandler/Code/S3Bucket", + "resourecType": "AWS::S3::Bucket", + "metadata": [ + { + "sourceLocation": "sample-source-location" + } + ] + }, + "S3VersionKey": { + "path": "CdkWorkshopStack/HelloHandler/Code/S3VersionKey", + "metadata": [ + { + "sourceLocation": "sample-source-location" + } + ] + }, + "AssetBucket": { + "path": "CdkWorkshopStack/HelloHandler/Code/AssetBucket", + "metadata": [ + { + "sourceLocation": "sample-source-location" + } + ], + "children": { + "Resource": { + "path": "CdkWorkshopStack/HelloHandler/ServiceRole/Resource", + "metadata": [ { - "id": "ServiceRole", - "path": "CdkWorkshopStack/HelloHandler/ServiceRole", - "metadata": [ - { - "sourceLocation": "sample-source-location", - "properties": "...properties" - }, - { - "id": "Resource", - "path": "CdkWorkshopStack/HelloHandler/Resource", - "metadata": [ - { - "sourceLocation": "sample-source-location", - "properties": [ - { - "logicalId": "HelloHandler2E4FBA4D" - } - ] - } - ] - }, - { - "id": "Code", - "path": "CdkWorkshopStack/HelloHandler/Code", - "metadata": [ - { - "sourceLocation": "sample-source-location" - } - ], - "children": [ - { - "id": "S3Bucket", - "path": "CdkWorkshopStack/HelloHandler/Code/S3Bucket", - "resourecType": "AWS::S3::Bucket", - "metadata": [ - { - "sourceLocation": "sample-source-location" - } - ] - }, - { - "id": "S3VersionKey", - "path": "CdkWorkshopStack/HelloHandler/Code/S3VersionKey", - "metadata": [ - { - "sourceLocation": "sample-source-location" - } - ] - }, - { - "id": "AssetBucket", - "path": "CdkWorkshopStack/HelloHandler/Code/AssetBucket", - "metadata": [ - { - "sourceLocation": "sample-source-location" - } - ] - } - ] - } - ], - "children": [ - { - "id": "Resource", - "path": "CdkWorkshopStack/HelloHandler/ServiceRole/Resource", - "metadata": [ - { - "sourceLocation": "sample-source-location", - "properties": [ - { - "logicalId": "HelloHandlerServiceRole11EF7C63" - } - ] - } - ] - } - ] + "sourceLocation": "sample-source-location", + "properties": { + "logicalId": "HelloHandlerServiceRole11EF7C63" + } } - ] + ] + } + } } - ] + } + } + } } - ] + } + } + } } + } } ``` ### Construct metadata -We will also introduce an interface that constructs *can* implement to provide metadata such as properties, +We will also introduce an interface that constructs *can* implement to provide metadata such as configured values, additional context, anything a construct would want to report back as properties. Constructs will be able to supply information about themselves for inclusion into the context tree. @@ -317,10 +300,12 @@ export interface IDisplayable { ## Unresolved Questions * Experience across all supported languages - nuances, considerations, etc +* How to report on stability of properties that are rendered and returned through "IDisplayable" ## Future Possibilities -We do not expect customers to have to interact directly with the outputs that the CDK produces as a part of the cloud assembly. Publishing the CDK construct tree as an output to the cloud assembly is intended to be a starting point to build on and create experiences based on it. +We do not expect customers to have to interact directly with the outputs that the CDK produces as a part of the cloud assembly. +Publishing the CDK construct tree as an output to the cloud assembly is intended to be a starting point to build on and create experiences based on it. ### Tooling @@ -329,3 +314,5 @@ With a construct tree model in place and construct tree being published as a `.j **Diff** diff the construct tree against the resources that are deployed in the cloud. This capability would require cloud capability but we should always be able to render a construct tree without the need to connect. **Navigability** include the location in code where the construct is defined. This would start to pave the way towards building tooling and extensions that IDE's can leverage. + +**Errors/Warnings** constructs can report errors/warnings as metadata. Surfacing this information through tooling would be helpful in resolving problems when constructing applications with the CDK. From 830c13a7fb6a1058037041600586ab9a3b642e63 Mon Sep 17 00:00:00 2001 From: Shiv Lakshminarayan Date: Thu, 17 Oct 2019 03:32:36 -0700 Subject: [PATCH 8/9] moving errors/warnings as additional functionality. filling in source columns for construct properties --- design/construct-tree.md | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-) diff --git a/design/construct-tree.md b/design/construct-tree.md index 6af6e1f859abc..cc7645dd5efef 100644 --- a/design/construct-tree.md +++ b/design/construct-tree.md @@ -106,7 +106,6 @@ This list describes only the minimal set of requirements from this feature. Afte 3. ***Metadata -*** Constructs will expose metadata which will include an array of objects associated with the construct. 1. **Location** - The full path of the location where the construct is defined. This will support use cases where the construct-tree nodes can support navigability back to the declaration in the IDE itself. 2. ***Properties*** - Constructs will opt into the set of properties, and these will be exposed. Higher level constructs (L2 and L3) will expose properties and values of the resources they contain and not necessarily enumerate every property that’s available for a resource. This will especially be useful for constructs that are opinionated and set defaults on behalf of developers. Low level constructs (L1) will expose all properties that are configured. - 3. ***Errors/Warnings*** - Errors and warnings that are produced during synthesis indicating validation failures, deprecation notices, guidance, etc will be included in the tree. These are emitted at the construct level although the message may point towards a specific property. The construct tree should be complete and include validation failures for all resources in the CDK application rather than just just fail on the first error. 4. **Assets** - Assets information will be included in the construct tree. Assets represent actions that the CDK *will* take ahead of stack deployments (i.e. S3 assets are zipped and uploaded directly, Docker images are uploaded to ECR). The asset metadata in the tree is the S3 key that an asset *would* have 3. **Local** - The construct tree is produced at synthesis time. Connectivity to AWS and CloudFormation should not be required to produce a tree view of a CDK application. @@ -122,21 +121,21 @@ The construct tree will be a list of paths that are indexed into a map of constr ### Construct properties -|Property |Type |Required |Source | Description | -|--- |--- |--- |--- | --- | -|path |string |Required |TODO | full, absolute path of the construct within the tree | -|children |Array |Not Required |TODO | All direct children of this construct. Array of the absolute paths of the constructs. Will be used to walk entire list of constructs | -|metadata |Array |Not Required |TODO | Metadata describing all constructs/resources that are encapsulated by the construct | +|Property |Type |Required |Source | Description | +|--- |--- |--- |--- | --- | +|path |string |Required |`construct.node.path` | Full, absolute path of the construct within the tree | +|children |Array |Not Required |`construct.node.children` | All direct children of this construct. Array of the absolute paths of the constructs. Will be used to walk entire list of constructs | +|metadata |Array |Not Required |`construct.node.metadata` | Metadata describing all constructs/resources that are encapsulated by the construct | ### Metadata Properties The following metadata properties will be included by the construct that produces the `tree.json` output. -|Property |Type |Required |Source | Description | -|--- |--- |--- |--- | --- | -|sourceLocation |string |Required |TODO | location in source code where the construct is defined | -|description |string |Not Required |TODO | description of the construct | -|properties |Array |Not Required |TODO | constructs can fill in arbitrary metadata such as configuration. CloudFormation agnostic. CFN constructs will include the properties in `this.toCloudFormation()` | +|Property |Type |Required | Description | +|--- |--- |--- | --- | +|sourceLocation |string |Required | location in source code where the construct is defined | +|description |string |Not Required | description of the construct | +|properties |Array |Not Required | constructs can fill in arbitrary metadata such as configuration. CloudFormation agnostic. CFN constructs will include the properties in `this.toCloudFormation()` | **TODO** - add detail and a more concrete walkthrough with samples of all the attributes referenced in construct and metadata properties. @@ -315,4 +314,4 @@ With a construct tree model in place and construct tree being published as a `.j **Navigability** include the location in code where the construct is defined. This would start to pave the way towards building tooling and extensions that IDE's can leverage. -**Errors/Warnings** constructs can report errors/warnings as metadata. Surfacing this information through tooling would be helpful in resolving problems when constructing applications with the CDK. +**Errors/Warnings** - Errors and warnings that are produced during synthesis indicating validation failures, deprecation notices, guidance, etc will be included in the tree. These are emitted at the construct level although the message may point towards a specific property. The construct tree should be complete and include failures for all resources in the CDK application rather than just just fail on the first error. From 91914166413b9ad71a600905552c445812800fcc Mon Sep 17 00:00:00 2001 From: Shiv Lakshminarayan Date: Tue, 5 Nov 2019 00:39:42 -0800 Subject: [PATCH 9/9] simplify and rename attributes interface. Updated tree to share type and properties by naming convention --- design/construct-tree.md | 178 ++++++++++----------------------------- 1 file changed, 44 insertions(+), 134 deletions(-) diff --git a/design/construct-tree.md b/design/construct-tree.md index cc7645dd5efef..9c9c44c327f74 100644 --- a/design/construct-tree.md +++ b/design/construct-tree.md @@ -64,14 +64,12 @@ In this scenario, these are some details that are not intuitive without diving d >2. What properties were the resources initialized with? Can they be changed? >3. Is the relationship between these constructs captured in the cloud assembly correctly? - When deployments fail, or provisioned resources don’t hold the properties that one might expect, developers can’t figure out what properties that resources such as `NatGateway, AutoScalingGroup, Subnets, Route Tables` were configured with without dropping into the generated CloudFormation template. It can take a good amount of clicking and digging before developers can figure out what happened and almost certainly needs them to get away from their favourite IDE and switch context to find out. The recurring theme is that intent based constructs are easy to get started with, take a lot of complexity away from the developer. The flip-side of this experience is that debugging and figuring out what the CDK did on your behalf can become challenging as applications come more complex. This problem will proliferate as more third-party constructs become available for consumption. Exposing the construct tree will enrich the developer experience for CDK application construction by exposing the construct tree explicitly. - Let’s take another look at an example of what a construct tree might look like: This is a Lambda function that has it’s handler code written as an asset: @@ -121,11 +119,11 @@ The construct tree will be a list of paths that are indexed into a map of constr ### Construct properties -|Property |Type |Required |Source | Description | -|--- |--- |--- |--- | --- | -|path |string |Required |`construct.node.path` | Full, absolute path of the construct within the tree | -|children |Array |Not Required |`construct.node.children` | All direct children of this construct. Array of the absolute paths of the constructs. Will be used to walk entire list of constructs | -|metadata |Array |Not Required |`construct.node.metadata` | Metadata describing all constructs/resources that are encapsulated by the construct | +|Property |Type |Required |Source | Description | +|--- |--- |--- |--- | --- | +|path |string |Required |`construct.node.path` | Full, absolute path of the construct within the tree | +|children |Array |Not Required |`construct.node.children` | All direct children of this construct. Array of the absolute paths of the constructs. Will be used to walk entire list of constructs | +|attributes |Array |Not Required |`construct.node.attributes` | Attributes describing all constructs/resources/properties that are encapsulated by the construct | ### Metadata Properties @@ -133,129 +131,50 @@ The following metadata properties will be included by the construct that produce |Property |Type |Required | Description | |--- |--- |--- | --- | -|sourceLocation |string |Required | location in source code where the construct is defined | -|description |string |Not Required | description of the construct | -|properties |Array |Not Required | constructs can fill in arbitrary metadata such as configuration. CloudFormation agnostic. CFN constructs will include the properties in `this.toCloudFormation()` | +|attributes |Array |Not Required | constructs can fill in arbitrary metadata such as configuration, type, properties, etc | + +Attributes are an extensible list and their keys should be namespaced by convention to avoid conflicts. -**TODO** - add detail and a more concrete walkthrough with samples of all the attributes referenced in construct and metadata properties. +As an example, L1 CloudFormation resources would use the following convention within the bag of attributes: +`aws:cdk:cloudformation:type` - CloudFormation resource that this construct represents +`aws:cdk:cloudformation:properties` - properties of the CloudFormation resource + +The example below is a sample tree with a stack that has 2 resources within it. The resources have attributes in their bag that are named by convention. ```json { - "version": "1.0-experimental", + "version": "tree-0.1", "tree": { - "App": { - "path": "/", - "metadata": { - "sourceLocation": "" + "id": "App", + "path": "", + "children": { + "Tree": { + "path": "Tree" }, - "children": { - "CdkWorkshopStack": { - "path": "CdkWorkshopStack", - "metadata": [ - { - "sourceLocation": "/lib/cdkworkshop-stack.ts:8:22", - "description": "Construct for a CloudFormation Stack", - "properties": { - "notificationArn": "sample-notification-arn", - "retentionPolicy": "RETAIN", - "role-arn": "sample-role-arn" - }, - "warnings": [ - { - "deprecatedParameter": "`transform` is deprecated, use `transforms` instead" + "mystack": { + "path": "mystack", + "children": { + "mycfnresource": { + "path": "mystack/mycfnresource", + "attributes": { + "aws:cdk:props": { + "mystringpropkey": "mystringpropval", + "mylistpropkey": [ + "listitem1" + ], + "mystructpropkey": { + "myboolpropkey": true, + "mynumpropkey": 50 } - ] + } } - ], - "children": { - "HelloHandler": { - "path": "CdkWorkshopStack/HelloHandler", - "metadata": [ - { - "sourceLocation": "sample-source-location", - "properties": { - "memorySize": "128MB", - "Role": "sample-role", - "Tags": [ - { - "sample-tag": "yay-cdk", - "safe-to-delete": "yup" - } - ] - } - } - ], - "children": { - "ServiceRole": { - "path": "CdkWorkshopStack/HelloHandler/ServiceRole", - "metadata": [ - { - "sourceLocation": "sample-source-location", - "properties": { - "something-cool": "about-this-role" - } - } - ] - }, - "Resource": { - "path": "CdkWorkshopStack/HelloHandler/Resource", - "metadata": [ - { - "sourceLocation": "sample-source-location", - "properties": { - "logicalId": "HelloHandler2E4FBA4D" - } - } - ] - }, - "Code": { - "path": "CdkWorkshopStack/HelloHandler/Code", - "metadata": [ - { - "sourceLocation": "sample-source-location" - } - ], - "children": { - "S3Bucket": { - "path": "CdkWorkshopStack/HelloHandler/Code/S3Bucket", - "resourecType": "AWS::S3::Bucket", - "metadata": [ - { - "sourceLocation": "sample-source-location" - } - ] - }, - "S3VersionKey": { - "path": "CdkWorkshopStack/HelloHandler/Code/S3VersionKey", - "metadata": [ - { - "sourceLocation": "sample-source-location" - } - ] - }, - "AssetBucket": { - "path": "CdkWorkshopStack/HelloHandler/Code/AssetBucket", - "metadata": [ - { - "sourceLocation": "sample-source-location" - } - ], - "children": { - "Resource": { - "path": "CdkWorkshopStack/HelloHandler/ServiceRole/Resource", - "metadata": [ - { - "sourceLocation": "sample-source-location", - "properties": { - "logicalId": "HelloHandlerServiceRole11EF7C63" - } - } - ] - } - } - } - } - } + }, + "mytopic": { + "path": "mystack/mytopic", + "attributes": { + "aws:cdk:cloudformation:type": "AWS::SNS::Topic", + "aws:cdk:cloudformation:properties": { + "DisplayName": "MyTopic" } } } @@ -278,21 +197,12 @@ that have additional information to contribute to the construct tree : ```typescript /*** - * Displayable information that a construct will contribute towards the context tree* + * Information that a construct will contribute towards the construct tree* * such as dependencies, properties, links, documentation, etc */ -export interface IDisplayable { - - // id of the construct within the current scope - readonly displayName: string - - // Other displayable attributes +export interface ITreeAttributes { + // attributes will be added to the construct tree readonly attributes: [key: string]: string; - - /* method that supports extensibility on what can be displayed without - * adding more methods/properties on IDisplayable - */ - inspect(display: IDisplayer): void } ```