feat: publish construct hierarchy with metadata to cloud assembly #4194

nija-at · 2019-09-22T15:17:41Z

Publish the CDK construct hierarchy with the construct id and path to
the cloud assembly directory, with a corresponding manifest entry.

Motivation and proposal documented in RFC - #4053.

This change focuses on integrating generating the metadata into the
cloud assembly as part of the the 'synthesize' lifecycle phase.
Subsequent changes will enrich the metadata file with additional
information from the construct tree (see RFC).

BREAKING CHANGES:

environment is no longer a property of CloudArtifact, instead is a property of its subclass - CloudformationArtifact.
construct.node.setContext() no longer works on the App construct. Use AppProps instead to pass context values.

The following CDK app generates the subsequent tree metadata JSON file

CDK app:

const app = new cdk.App();
const stack = new cdk.Stack(app, 'mystack');

const topic = new sns.Topic(this, 'mytopic');
const queue = new sqs.Queue(this, 'myqueue');
topic.addSubscription(new snssubs.SqsSubscription(queue));

app.synth();

Construct tree JSON:

{
  "version": "tree-0.1",
  "tree": {
    "id": "App",
    "path": "",
    "children": [
      {
        "id": "mystack",
        "path": "mystack",
        "children": [
          {
            "id": "mytopic",
            "path": "mystack/mytopic",
            "children": [
              {
                "id": "Resource",
                "path": "mystack/mytopic/Resource"
              }
            ]
          },
          {
            "id": "myqueue",
            "path": "mystack/myqueue",
            "children": [
              {
                "id": "Resource",
                "path": "mystack/myqueue/Resource"
              },
              {
                "id": "Policy",
                "path": "mystack/myqueue/Policy",
                "children": [
                  {
                    "id": "Resource",
                    "path": "mystack/myqueue/Policy/Resource"
                  }
                ]
              },
              {
                "id": "mystackmytopic28FA060E",
                "path": "mystack/myqueue/mystackmytopic28FA060E",
                "children": [
                  {
                    "id": "Resource",
                    "path": "mystack/myqueue/mystackmytopic28FA060E/Resource"
                  }
                ]
              }
            ]
          }
        ]
      },
      {
        "id": "ConstructTreeMetadata",
        "path": "ConstructTreeMetadata"
      }
    ]
  }
}

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

Publish the CDK construct hierarchy with the construct id and path to the cloud assembly directory, with a corresponding manifest entry. Motivation and proposal documented in RFC - #4053. This change focuses on integrating generating the metadata into the cloud assembly as part of the the 'synthesize' lifecycle phase. Subsequent changes will enrich the metadata file with additional information from the construct tree (see RFC).

mergify · 2019-09-22T15:18:17Z

Thanks so much for taking the time to contribute to the AWS CDK ❤️

We will shortly assign someone to review this pull request and help get it
merged. In the meantime, please take a minute to make sure you follow this
checklist:

PR title type(scope): text
- type: fix, feat, refactor go into CHANGELOG, chore is hidden
- scope: name of module without aws- or cdk- prefix or postfix (e.g. s3 instead of aws-s3-deployment)
- text: use all lower-case, do not end with a period, do not include issue refs
PR Description
- Rationale: describe rationale of change and approach taken
- Issues: indicate issues fixed via: fixes #xxx or closes #xxx
- Breaking?: last paragraph: BREAKING CHANGE: <describe what changed + link for details>
Testing
- Unit test added. Prefer to add a new test rather than modify existing tests
- CLI or init templates change? Re-run/add CLI integration tests
Documentation
- README: update module README to describe new features
- API docs: public APIs must be documented. Copy from official AWS docs when possible
- Design: for significant features, follow design process

aws-cdk-automation · 2019-09-22T15:21:04Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

eladb

Add an example output in the PR description

packages/@aws-cdk/core/lib/private/construct-tree-metadata.ts

packages/@aws-cdk/core/lib/app.ts

nija-at · 2019-09-22T15:42:17Z

Note, the inclusion of a BREAKING CHANGE in the PR description.

aws-cdk-automation · 2019-09-22T15:42:46Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2019-09-22T15:59:50Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2019-09-22T17:47:52Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

packages/@aws-cdk/core/lib/app.ts

* Rename ConstructTreeMetadata to Tree * Wrap tree construct with a version number * Move Tree construct into the constructor

aws-cdk-automation · 2019-09-23T15:51:58Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2019-09-23T15:55:45Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2019-09-23T16:52:38Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2019-09-24T08:46:34Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2019-09-24T08:48:59Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

packages/@aws-cdk/core/lib/private/tree.ts

eladb · 2019-09-24T09:55:54Z

packages/@aws-cdk/core/lib/app.ts

+
+  /**
+   * Include construct tree metadata as part of the Cloud Assembly.
+   *


Add a "@see" with a link to the RFC

Sure. I'll wait until the RFC is published so I can get the a more stable link.

packages/@aws-cdk/core/lib/app.ts

packages/@aws-cdk/core/lib/private/tree.ts

packages/@aws-cdk/core/test/test.app.ts

packages/@aws-cdk/cx-api/lib/cloud-artifact.ts

packages/@aws-cdk/cx-api/lib/cloud-assembly.ts

packages/@aws-cdk/cx-api/lib/cloudformation-artifact.ts

* Renamed Tree to TreeMetadata * Re-typed 'metadata' to 'tree' in CloudArtifacts

aws-cdk-automation · 2019-09-24T14:44:27Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2019-09-24T14:45:27Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2019-09-25T08:13:41Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

eladb · 2019-09-26T13:02:38Z

packages/@aws-cdk/cx-api/lib/cloud-artifact.ts

@@ -80,7 +84,8 @@ export class CloudArtifact {
    switch (artifact.type) {
      case ArtifactType.AWS_CLOUDFORMATION_STACK:
        return new CloudFormationStackArtifact(assembly, id, artifact);
-
+      case ArtifactType.CDK_TREE:
+        return new TreeCloudArtifact(assembly, id, artifact);
      default:
        throw new Error(`unsupported artifact type: ${artifact.type}`);


IMPORTANT: Just realized that this change requires a bump in the cx-protocol version because we were stupid enough to throw an error when reading a cloud assembly with unrecognized artifact types (no future compatibility). This means that an old CLI will crash if it will try to synth a CDK app that has this change.

Please also remove this default: section and simply ignore unknown artifacts. That should be the correct behavior going forward so we can avoid such issues.

At least make it a warning or something right. Not srue that ignoring is the right behavior.

The warning should say something like "Unrecognized artifact type: ${type}. You need to upgrade your CDK CLI to use this artifact" or something. Give users a hint on how to remedy this.

I don't think it makes total sense to issue a warning from this library. Maybe the CLI can issue a warning if this returns undefined

Since we've added a new type of artifact to the cloud assembly, namely CDK_TREE, the protocol version needs to be bumped. This will result in CDK customers being notified to upgrade the CDK CLI to >= 1.10.0. To prevent this from occurring in the future, the change also changes the behaviour when an unknown artifact type is detected when reading the cloud assembly manifest. This ensures that the CLI will be forwards compatible to cloud assemblies when new artifact types are added to them. See #4194 (comment)

Since we've added a new type of artifact to the cloud assembly, namely CDK_TREE, the protocol version needs to be bumped. This will result in CDK customers being notified to upgrade the CDK CLI to >= 1.10.0. To prevent this from occurring in the future, the change also changes the behaviour when an unknown artifact type is detected when reading the cloud assembly manifest. This ensures that the CLI will be forwards compatible to cloud assemblies when new artifact types are added to them. See aws#4194 (comment)

nija-at requested review from rix0rrr, eladb and shivlaks September 22, 2019 15:17

nija-at self-assigned this Sep 22, 2019

Trying to fix environment

12ffcda

eladb reviewed Sep 22, 2019

View reviewed changes

Fix breaking tests in @aws-cdk/assets

4005ea7

Niranjan Jayakar added 2 commits September 22, 2019 18:05

Option to disable construct tree gen

6254c2b

Prepare only once

4e46368

eladb reviewed Sep 23, 2019

View reviewed changes

packages/@aws-cdk/core/lib/app.ts Outdated Show resolved Hide resolved

PR feedback

bf942ad

* Rename ConstructTreeMetadata to Tree * Wrap tree construct with a version number * Move Tree construct into the constructor

nija-at added the pr/do-not-merge This PR should not be merged at this time. label Sep 23, 2019

Merge branch 'master' into nija-at/app-metadata

c8aba4c

Fix broken route53 tests

cc6d4a0

Better fix for failing Route53 tests

56e2af3

nija-at requested a review from eladb September 24, 2019 08:08

Merge branch 'master' into nija-at/app-metadata

ce6bca2

eladb suggested changes Sep 24, 2019

View reviewed changes

Niranjan Jayakar and others added 2 commits September 24, 2019 15:03

PR feedback

8f7d849

* Renamed Tree to TreeMetadata * Re-typed 'metadata' to 'tree' in CloudArtifacts

Merge branch 'master' into nija-at/app-metadata

a8c6b09

nija-at requested a review from eladb September 24, 2019 14:04

eladb approved these changes Sep 25, 2019

View reviewed changes

Merge branch 'master' into nija-at/app-metadata

7364581

nija-at merged commit 3cca03d into master Sep 25, 2019

nija-at deleted the nija-at/app-metadata branch September 25, 2019 08:29

eladb reviewed Sep 26, 2019

View reviewed changes

nija-at mentioned this pull request Sep 26, 2019

chore(cx-api): bump CLOUD_ASSEMBLY_VERSION to 1.10.0 #4259

Merged

NGL321 added the contribution/core This is a PR that came from AWS. label Sep 27, 2019

jogold mentioned this pull request Oct 3, 2019

Cannot set context at app level after v1.9.0 #4359

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: publish construct hierarchy with metadata to cloud assembly #4194

feat: publish construct hierarchy with metadata to cloud assembly #4194

nija-at commented Sep 22, 2019 •

edited

Loading

mergify bot commented Sep 22, 2019

aws-cdk-automation commented Sep 22, 2019

eladb left a comment

nija-at commented Sep 22, 2019

aws-cdk-automation commented Sep 22, 2019

aws-cdk-automation commented Sep 22, 2019

aws-cdk-automation commented Sep 22, 2019

aws-cdk-automation commented Sep 23, 2019

aws-cdk-automation commented Sep 23, 2019

aws-cdk-automation commented Sep 23, 2019

aws-cdk-automation commented Sep 24, 2019

aws-cdk-automation commented Sep 24, 2019

eladb Sep 24, 2019

nija-at Sep 24, 2019 •

edited

Loading

aws-cdk-automation commented Sep 24, 2019

aws-cdk-automation commented Sep 24, 2019

aws-cdk-automation commented Sep 25, 2019

eladb Sep 26, 2019

rix0rrr Sep 26, 2019

rix0rrr Sep 26, 2019

eladb Sep 26, 2019

feat: publish construct hierarchy with metadata to cloud assembly #4194

feat: publish construct hierarchy with metadata to cloud assembly #4194

Conversation

nija-at commented Sep 22, 2019 • edited Loading

mergify bot commented Sep 22, 2019

aws-cdk-automation commented Sep 22, 2019

AWS CodeBuild CI Report

eladb left a comment

Choose a reason for hiding this comment

nija-at commented Sep 22, 2019

aws-cdk-automation commented Sep 22, 2019

AWS CodeBuild CI Report

aws-cdk-automation commented Sep 22, 2019

AWS CodeBuild CI Report

aws-cdk-automation commented Sep 22, 2019

AWS CodeBuild CI Report

aws-cdk-automation commented Sep 23, 2019

AWS CodeBuild CI Report

aws-cdk-automation commented Sep 23, 2019

AWS CodeBuild CI Report

aws-cdk-automation commented Sep 23, 2019

AWS CodeBuild CI Report

aws-cdk-automation commented Sep 24, 2019

AWS CodeBuild CI Report

aws-cdk-automation commented Sep 24, 2019

AWS CodeBuild CI Report

eladb Sep 24, 2019

Choose a reason for hiding this comment

nija-at Sep 24, 2019 • edited Loading

Choose a reason for hiding this comment

aws-cdk-automation commented Sep 24, 2019

AWS CodeBuild CI Report

aws-cdk-automation commented Sep 24, 2019

AWS CodeBuild CI Report

aws-cdk-automation commented Sep 25, 2019

AWS CodeBuild CI Report

eladb Sep 26, 2019

Choose a reason for hiding this comment

rix0rrr Sep 26, 2019

Choose a reason for hiding this comment

rix0rrr Sep 26, 2019

Choose a reason for hiding this comment

eladb Sep 26, 2019

Choose a reason for hiding this comment

nija-at commented Sep 22, 2019 •

edited

Loading

nija-at Sep 24, 2019 •

edited

Loading