Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rfc: changes to default stack environments #4131

Closed
wants to merge 4 commits into from

Conversation

eladb
Copy link
Contributor

@eladb eladb commented Sep 18, 2019

We are looking to change the default behavior for when a Stack is defined without env in order to improve the developer experience.

When using cdk deploy directly with an app that has stack(s) without explicit env, we can inherit the CLI's current environment instead of synthesize environment-agnostic stacks.

Comments are welcome!

Rendered view: here


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@mergify
Copy link
Contributor

mergify bot commented Sep 18, 2019

Thanks so much for taking the time to contribute to the AWS CDK ❤️

We will shortly assign someone to review this pull request and help get it
merged. In the meantime, please take a minute to make sure you follow this
checklist
:

  • PR title type(scope): text
    • type: fix, feat, refactor go into CHANGELOG, chore is hidden
    • scope: name of module without aws- or cdk- prefix or postfix (e.g. s3 instead of aws-s3-deployment)
    • text: use all lower-case, do not end with a period, do not include issue refs
  • PR Description
    • Rationale: describe rationale of change and approach taken
    • Issues: indicate issues fixed via: fixes #xxx or closes #xxx
    • Breaking?: last paragraph: BREAKING CHANGE: <describe what changed + link for details>
  • Testing
    • Unit test added. Prefer to add a new test rather than modify existing tests
    • CLI or init templates change? Re-run/add CLI integration tests
  • Documentation
    • README: update module README to describe new features
    • API docs: public APIs must be documented. Copy from official AWS docs when possible
    • Design: for significant features, follow design process

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

rix0rrr
rix0rrr previously approved these changes Sep 18, 2019
design/default-env.md Outdated Show resolved Hide resolved
design/default-env.md Show resolved Hide resolved
@rix0rrr rix0rrr added the pr/do-not-merge This PR should not be merged at this time. label Sep 18, 2019
@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@eladb eladb self-assigned this Sep 18, 2019
@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

- When `cdk synth` prints to STDOUT is prints in YAML (unless `--json` is specified), while `cdk.out` always uses JSON ([#2965](https://github.com/aws/aws-cdk/issues/2965)).
- `cdk.out` is created no matter what, for both use cases of `synth` and even if it the app was synthesized implicitly as part of `cdk deploy`. This also somewhat causes confusion.

We propose to separate the "synth" use cases into two different CLI commands: `cdk print` and `cdk synth`. The first will serve the inspection use case and the second the CI/CD use case. This way, we can decide that `--default-env` will have a different default behavior for these two commands. For `cdk print` we will default to `agnostic` and for `cdk synth` we will default to `explicit`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am worried that people would want to use cdk print (I don't love the name here, BTW) to check what will be deployed, and that they will be surprised that what is synth'd and deploy'd is not the same...

To me, literally the only reason why you'd want to "see the template" is to troubleshoot or validate what'll be deployed before you go do it... If the default behaviors aren't the same, then this will be surprising.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we even need the print subcommand. A combination of good output messaging on cdk synth (like "synthesized stackA at ") and cat/less/vim/etc. should work well just fine here.

That would also mean that synth acknowledge the --json (or yaml) option and render the stack under cdk.out appropriately.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would comment that the default environment for cdk deploy and cdk print would need to be the same for print to be useful (if it is intended for debugging). However, IF it is intended for debugging, we'd be better off calling it something else cdk debug? cdk troubleshoot? Or make it a flag to synth: cdk synth --as-deployed.

@pgarbe
Copy link
Contributor

pgarbe commented Sep 19, 2019

After reading it, my first impression was that there are far too many options which makes it very complex. How am I supposed to explain this to someone who has little or no knowledge of CDK? It get's even more complicated if you think about the "lookupFromXXX()" methods. And how does the cdk package command play into it?

@eladb
Copy link
Contributor Author

eladb commented Sep 22, 2019

@pgarbe yes, this could be quite confusing, but the purpose here is to actually make the experience more seamless. For example, at the moment if you want to use lookupFromXxx you have to define your stack with an explicit env, which is redundant if you are using cdk deploy and we know exactly where this stack is going.

But I hear you... And I agree with @RomainMuller's point that it might be confusing for people that use cdk print and cdk deploy. How about we make both cdk print and cdk deploy to default to --default-env=inherit. This at least will make the "interactive" use case consistent. And then cdk synth will have --default-env=explicit.

@pgarbe
Copy link
Contributor

pgarbe commented Sep 23, 2019

@eladb Yes, I agree that the current situation with lookupFromXxx is also confusing. I guess it needs more communication about definition of environment-agnostic and not environment-agnostic stacks, when to use what and the consequences.

@NGL321 NGL321 added the contribution/core This is a PR that came from AWS. label Sep 23, 2019
@eladb eladb removed their assignment Oct 27, 2019
@eladb eladb changed the title feat(core): changes to default stack environments rfc: changes to default stack environments Nov 10, 2019
@eladb eladb requested a review from a team November 13, 2019 19:02
Copy link
Contributor

@MrArnoldPalmer MrArnoldPalmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Though I agree that the number of options creates some complexity, it is my feeling that this complexity exists within the underlying tools (aws-cli, cloudformation, etc) with regards to regions. Overall I believe this proposal would indeed make usage more intuitive for the synth and deploy workflows for the default case (not passing a --default-env).

Separation of the print command also makes sense to me. A common alternative is using an option --dry-run that when enabled would print the template to stdout instead of creating .cdk.out/. For those that believe that synth should cover the same use case this may make more sense.

@eladb eladb added the management/rfc request for comments label Nov 13, 2019
The proposal is:

1. Add an `--default-env=inherit|explicit|agnostic` switch to the CLI which will control the default env mode.
2. Extend the semantics of `CDK_DEFAULT_ACCOUNT` and `CDK_DEFAULT_REGION` to be able to express these three modes (currently it cannot express environment-agnosticness).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a vague sentence. Does that mean the values in those environment variables will be different? What do you mean here?

3. Change the default behavior of `env` in `Stack` to fall back to `CDK_DEFAULT_ACCOUNT` and `CDK_DEFAULT_REGION` if an explicit value is not specified.
4. If `env` is not explicitly set and `CDK_DEFAULT_XXX` is undefined, stack initialization **will fail with an error**.
5. The default mode for `cdk deploy` will be `inherit`. This is because when a stack without env is deployed with the CLI, we *know* where it's going. There's no point playing this game here.
6. The default mode for `cdk synth` will be `explicit`. The implication of this (detalied below) is that if users wish to synthesize an artifact from their CDK app and deploy it at a later stage, they will have to either explicitly specify `env` for all stacks or use `cdk synth --default-env=inherit|agnostic`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find this dangerous, and we should not treat the behavior change lightly. I think people have been trained to look at cdk synth to determine what their deployed stack will look like. I know I do.

We HAVE TO commit to clearly documenting/explaining how you troubleshoot your deployment (look at the generated template).

I would also say we have to have cdk synth detect whether a cdk deploy would deploy something else and printing something like:

The stack you see above is environment-agnostic and may not be what will be deployed when you run `cdk deploy`.
See https://docs.aws.amazon.com/.../troubleshooting.html

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also the "environments" section of the developer guide will need be rewritten. Ping @jerry-aws .

- When `cdk synth` prints to STDOUT is prints in YAML (unless `--json` is specified), while `cdk.out` always uses JSON ([#2965](https://github.com/aws/aws-cdk/issues/2965)).
- `cdk.out` is created no matter what, for both use cases of `synth` and even if it the app was synthesized implicitly as part of `cdk deploy`. This also somewhat causes confusion.

We propose to separate the "synth" use cases into two different CLI commands: `cdk print` and `cdk synth`. The first will serve the inspection use case and the second the CI/CD use case. This way, we can decide that `--default-env` will have a different default behavior for these two commands. For `cdk print` we will default to `agnostic` and for `cdk synth` we will default to `explicit`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would comment that the default environment for cdk deploy and cdk print would need to be the same for print to be useful (if it is intended for debugging). However, IF it is intended for debugging, we'd be better off calling it something else cdk debug? cdk troubleshoot? Or make it a flag to synth: cdk synth --as-deployed.

@mergify
Copy link
Contributor

mergify bot commented Dec 16, 2019

Thanks so much for taking the time to contribute to the AWS CDK ❤️

We will shortly assign someone to review this pull request and help get it
merged. In the meantime, please take a minute to make sure you follow this
checklist
:

  • PR title type(scope): text
    • type: fix, feat, refactor go into CHANGELOG, chore is hidden
    • scope: name of module without aws- or cdk- prefix or postfix (e.g. s3 instead of aws-s3-deployment)
    • text: use all lower-case, do not end with a period, do not include issue refs
  • PR Description
    • Rationale: describe rationale of change and approach taken
    • Issues: indicate issues fixed via: fixes #xxx or closes #xxx
    • Breaking?: last paragraph: BREAKING CHANGE: <describe what changed + link for details>
  • Testing
    • Unit test added. Prefer to add a new test rather than modify existing tests
    • CLI or init templates change? Re-run/add CLI integration tests
  • Documentation
    • README: update module README to describe new features
    • API docs: public APIs must be documented. Copy from official AWS docs when possible
    • Design: for significant features, follow design process

3 similar comments
@mergify
Copy link
Contributor

mergify bot commented Dec 16, 2019

Thanks so much for taking the time to contribute to the AWS CDK ❤️

We will shortly assign someone to review this pull request and help get it
merged. In the meantime, please take a minute to make sure you follow this
checklist
:

  • PR title type(scope): text
    • type: fix, feat, refactor go into CHANGELOG, chore is hidden
    • scope: name of module without aws- or cdk- prefix or postfix (e.g. s3 instead of aws-s3-deployment)
    • text: use all lower-case, do not end with a period, do not include issue refs
  • PR Description
    • Rationale: describe rationale of change and approach taken
    • Issues: indicate issues fixed via: fixes #xxx or closes #xxx
    • Breaking?: last paragraph: BREAKING CHANGE: <describe what changed + link for details>
  • Testing
    • Unit test added. Prefer to add a new test rather than modify existing tests
    • CLI or init templates change? Re-run/add CLI integration tests
  • Documentation
    • README: update module README to describe new features
    • API docs: public APIs must be documented. Copy from official AWS docs when possible
    • Design: for significant features, follow design process

@mergify
Copy link
Contributor

mergify bot commented Dec 16, 2019

Thanks so much for taking the time to contribute to the AWS CDK ❤️

We will shortly assign someone to review this pull request and help get it
merged. In the meantime, please take a minute to make sure you follow this
checklist
:

  • PR title type(scope): text
    • type: fix, feat, refactor go into CHANGELOG, chore is hidden
    • scope: name of module without aws- or cdk- prefix or postfix (e.g. s3 instead of aws-s3-deployment)
    • text: use all lower-case, do not end with a period, do not include issue refs
  • PR Description
    • Rationale: describe rationale of change and approach taken
    • Issues: indicate issues fixed via: fixes #xxx or closes #xxx
    • Breaking?: last paragraph: BREAKING CHANGE: <describe what changed + link for details>
  • Testing
    • Unit test added. Prefer to add a new test rather than modify existing tests
    • CLI or init templates change? Re-run/add CLI integration tests
  • Documentation
    • README: update module README to describe new features
    • API docs: public APIs must be documented. Copy from official AWS docs when possible
    • Design: for significant features, follow design process

@mergify
Copy link
Contributor

mergify bot commented Dec 16, 2019

Thanks so much for taking the time to contribute to the AWS CDK ❤️

We will shortly assign someone to review this pull request and help get it
merged. In the meantime, please take a minute to make sure you follow this
checklist
:

  • PR title type(scope): text
    • type: fix, feat, refactor go into CHANGELOG, chore is hidden
    • scope: name of module without aws- or cdk- prefix or postfix (e.g. s3 instead of aws-s3-deployment)
    • text: use all lower-case, do not end with a period, do not include issue refs
  • PR Description
    • Rationale: describe rationale of change and approach taken
    • Issues: indicate issues fixed via: fixes #xxx or closes #xxx
    • Breaking?: last paragraph: BREAKING CHANGE: <describe what changed + link for details>
  • Testing
    • Unit test added. Prefer to add a new test rather than modify existing tests
    • CLI or init templates change? Re-run/add CLI integration tests
  • Documentation
    • README: update module README to describe new features
    • API docs: public APIs must be documented. Copy from official AWS docs when possible
    • Design: for significant features, follow design process

@mergify
Copy link
Contributor

mergify bot commented Dec 16, 2019

Thanks so much for taking the time to contribute to the AWS CDK ❤️

We will shortly assign someone to review this pull request and help get it
merged. In the meantime, please take a minute to make sure you follow this
checklist
:

  • PR title type(scope): text
    • type: fix, feat, refactor go into CHANGELOG, chore is hidden
    • scope: name of module without aws- or cdk- prefix or postfix (e.g. s3 instead of aws-s3-deployment)
    • text: use all lower-case, do not end with a period, do not include issue refs
  • PR Description
    • Rationale: describe rationale of change and approach taken
    • Issues: indicate issues fixed via: fixes #xxx or closes #xxx
    • Breaking?: last paragraph: BREAKING CHANGE: <describe what changed + link for details>
  • Testing
    • Unit test added. Prefer to add a new test rather than modify existing tests
    • CLI or init templates change? Re-run/add CLI integration tests
  • Documentation
    • README: update module README to describe new features
    • API docs: public APIs must be documented. Copy from official AWS docs when possible
    • Design: for significant features, follow design process

1 similar comment
@mergify
Copy link
Contributor

mergify bot commented Dec 16, 2019

Thanks so much for taking the time to contribute to the AWS CDK ❤️

We will shortly assign someone to review this pull request and help get it
merged. In the meantime, please take a minute to make sure you follow this
checklist
:

  • PR title type(scope): text
    • type: fix, feat, refactor go into CHANGELOG, chore is hidden
    • scope: name of module without aws- or cdk- prefix or postfix (e.g. s3 instead of aws-s3-deployment)
    • text: use all lower-case, do not end with a period, do not include issue refs
  • PR Description
    • Rationale: describe rationale of change and approach taken
    • Issues: indicate issues fixed via: fixes #xxx or closes #xxx
    • Breaking?: last paragraph: BREAKING CHANGE: <describe what changed + link for details>
  • Testing
    • Unit test added. Prefer to add a new test rather than modify existing tests
    • CLI or init templates change? Re-run/add CLI integration tests
  • Documentation
    • README: update module README to describe new features
    • API docs: public APIs must be documented. Copy from official AWS docs when possible
    • Design: for significant features, follow design process

@mergify
Copy link
Contributor

mergify bot commented Dec 30, 2019

Title does not follow the guidelines of Conventional Commits. Please adjust title before merge.

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

eladb pushed a commit to aws/aws-cdk-rfcs that referenced this pull request Feb 3, 2020
Originally, we planned to disallow the use of assets by environment-agnostic stacks (e.g. stacks that are defined with pseudo ACCOUNT/REGION in `env`). The problem is that today this is the default behavior for `new Stack()`, which obviously can't work. The original plan was to modify the default behavior of `env` (see aws/aws-cdk#4131), and we still want to pursue this, but since this change in default behavior has potential risk involved, we wanted to decouple it from our CI/CD delivery.

Therefore, this change updates the CI/CD and cdk-assets RFCs to support assets in environment-agnostic stacks by allowing the use `${AWS::Region}` and `${AWS::AccountId}` in `assets.json`.
@eladb
Copy link
Contributor Author

eladb commented Mar 1, 2020

Since this is not in the critical path for CI/CD, I am closing for now and we will revisit soon

@eladb eladb closed this Mar 1, 2020
@rix0rrr rix0rrr deleted the benisrae/rfc/default-env branch April 12, 2021 09:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contribution/core This is a PR that came from AWS. management/rfc request for comments pr/do-not-merge This PR should not be merged at this time.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants