Document how to use stack parameters

[rclark]

Every example stack that I've seen so far in the documentation has no Parameters. I found the @aws-cdk/core documentation for the Parameter class itself, and got it to work in my stack (shows up in cdk synth output). Now, I don't know how to convey values for the parameters through cdk deploy.

❯ AWS_REGION=us-east-1 cdk deploy --GitSha abc
 ⏳  Starting deployment of stack hello-cdk...
 ❌  Deployment of stack hello-cdk failed: ValidationError: Parameters: [GitSha] must have values
Parameters: [GitSha] must have values

p.s. I also don't know where the hello-cdk name is coming from. I assume from the skeleton setup in cdk init?

[rix0rrr]

'hello-cdk' is the name that the Stack object gets constructed with.

Stack Parameters are currently not really in the path of how we're thinking about CDK apps (but admittedly, we're still looking for use cases). Instead, we encourage parameterizing the application and making the stacks as concrete as possible.

If you really have to use Stack Parameters, first of all please tell us more about your use case, and second of all, the workaround will be to synth your template to a file, then use AWS CLI or a different mechanism to upload it.

[rclark]

I see -- I do think there's still some gap that documentation needs a better bridge. A background concept of a cloudformation template as a declarative document clashes with trying to understand the CDK code as an "executable" where parameters would be provided to the program. I'm certainly still wrapping my head around this.

Instead, we encourage parameterizing the application and making the stacks as concrete as possible.

const app = new App(process.argv);

From the example. So I could use cdk deploy --with 'other' --arguments and parse the .argv. Is that how you'd propose I keep config separate from code?

The bummer about this is that as values for stack parameters, cloudformation describe-stacks API calls tell you about how the template has been configured. In this approach, you'd have to build your own system to keep track of configurations that were sent via application parameters.

That or read process.argv in order to populate values for @aws-cdk/core.Parameter objects within the application? Would that work?

[eladb]

Indeed, CloudFormation parameters are not the best way to convey degrees of freedom in CDK apps, since they are resolved only during deployment and therefore harder to reason about using “normal” code. This is why tactically we didn’t implement first class support for them yet in the toolkit. Having said that, I believe that if users wish to use them, understanding their limitations, it should be possible to pass in parameters in the toolkit when stacks are deployed.

Still, we don’t have good guidance for how to associate configuration to environments. In my mind the preferred mechanism would be to use per-environment context, which is a feature we have in our backlog and haven’t implemented yet.

The idea is as follows: when you define a stack, one of the props is called “env”. Today it allows you to explicitly specify “region” and “account”, but in the future it will simply be a string used as a key to a map within your “cdk.json” file. This per-environment map will be where you could define the environment (I.e account/region, but also using profiles, AWS Organizations, etc) and also associate context keys with values. Then, in your code, you’ll just call “construct.getContext(key)” to read these values when they are needed.

P.S. the context mechanism already exists, but at the moment is not associated with environment, so if you have multiple stacks you’ll need to organize the context keys to be able to distinguish between stacks.

I believe that this model, where config is source-controlled, and associated with a deployment environment, should fit the 12factor philosophy quite well.

Would love your thoughts on this approach.

[rclark]

in the future it will simply be a string used as a key to a map within your “cdk.json” file.

So basically you isolate config that may vary between deploys in the cdk.json file, correct? The only trouble with that model is that I believe the CDK application itself requires this file to be present in order to work at all. If that's true, then this cdk.json file will be something that's committed to version control alongside the application itself, and to me that's a violation of code/config separation.

A litmus test for whether an app has all config correctly factored out of the code is whether the codebase could be made open source at any moment, without compromising any credentials.

That's what's great about CloudFormation parameters -- as you say, "they are resolved only during deployment". I agree that this makes them harder to think about when you're writing a TypeScript application -- you find yourself having to keep a mental map in your head of which variables are "build time" (those that are resolved when the TypeScript app runs) vs. "deploy time" (those resolved by CloudFormation). You have to keep considering whether you access the values through CloudFormation intrinsic functions or not.

There's talk in the documentation about SSM Parameter Store. I looked at this service briefly for storing CloudFormation parameter values, but ended up moving past it, primarily because it required all values to be in plain text, which is not an option for sensitive credentials. Still, I wonder if the CDK use of parameter store is intended to help address these config/code differentiation issues in some way?

Another concept might be to make use of AWS Secrets Manager. Instead of storing my configuration in a local cdk.json file, could I store it in AWS Secrets Manager, and reference the SecretId in my cdk.json file per-environment? Certainly I could pull this off manually by using the aws-sdk to look up the configuration, but I wonder if the use-case would be worth more firm support in the CDK?

Even at that point, I'd still like to be able to pass command-line parameters through cdk deploy into my application. In our workflows, when you're running a deploy to some environment is the moment where you may wish to inject some change to the environment's configuration. Our internal deployment CLI does this by prompting you for CloudFormation parameter values. I suppose I could also inject that prompting into my CDK application code...

There is clearly more than one way to get this done -- and its also clearly a confusing shift for someone like me with well-established CloudFormation-based workflows.

[JMBreitenbach]

@rclark I completely agree with your statement 👍

So basically you isolate config that may vary between deploys in the cdk.json file, correct? The only trouble with that model is that I believe the CDK application itself requires this file to be present in order to work at all. If that's true, then this cdk.json file will be something that's committed to version control alongside the application itself, and to me that's a violation of code/config separation.

I'm really interested to hear about how best practice evolves around passing deployment config to the CDK apps.

We currently inject them at deployment using our CI pipe to inject the secrets in the CF vars. I can't actually see a way to keep the app 12 factor compatible without passing the args.

[JMBreitenbach]

I would expect the passing of deployment params to work something like the following:

[mneil]

I understand that ideally parameters would be added as configuration for most constructs. Create a pipeline in CDK and pass in the github repo, owner, and token (cdk.Secret) as parameters. When writing a TS application I also think that's a pretty simple way to deal with parameters.

However, Cloudformation is ~7 years old at this point and so we've already been using it for many years with workflows built around passing parameters to an entire stack (as opposed to an individual resource). I just ran into this issue:

I have an existing stack. I included it with cdk.include. This stack is huge and everything is interdependent (can't be broken down into smaller stacks). I need a way to pass parameters to this stack. As far as I can tell there's absolutely no way to do this. My hope was to use CDK to deploy this old stack then start writing newer stacks around it using CDK properly. Since I cannot pass any parameters to the stack I have to support a new workflow (CDK) and a legacy workflow. It would really help with adoption if it supported a more generic (even if it's inferior) way of using existing stacks and parameters.

I absolutely love that CDK can setup a stack with a bucket and push my stack to S3 before deploy. I love the progress output and events from CDK. I like that I can pick and choose stacks to deploy or deploy them all. Hopefully we can come up with some way to support existing workflows better.

[mneil]

Looking at the comment by @JMBreitenbach I just remembered that something along these lines was possible once. https://github.com/awslabs/aws-cdk/blame/aa76305132be01895d8b18f58085e8c9a7bab8a1/packages/@aws-cdk/cdk/lib/app.ts . I had an older version of CDK accepting input from argv. I don't think it would take in arbitrary stack parameters though. That would be a good spot to re-introduce this functionality. Or, perhaps, on the stack construct itself.

[Doug-AWS]

We have a section in the docs about passing in data:

https://awslabs.github.io/aws-cdk/passing-in-data.html

Feel free to re-open this issue if the docs do not satisfy your needs.

[seawatts]

Doug I'm still curious if it's possible to pass in cloudformation parameters in the cli or cdk.json just for testing purposes. So I can run cdk deploy locally.

[Doug-AWS]

Note that I've split the section up and moved it. I'm rebuilding the public docs now, so when I'm done I'll post a link to the new "How-Tos" section.

[Doug-AWS]

See https://docs.aws.amazon.com/CDK/latest/guide/passing_secrets_manager.html.

[evildani]

A common use case for passing parameters would be within service catalog, there is no other choice. If I want to write products in Service Catalog it is expected to provide parameters to cloudformation. Hopefully I make sense.

[joehillen]

@Doug-AWS Both those links are dead

[akirsman]

Why not providing a constructor overload such as public HelloStack(Construct parent, string id, IStackProps props, IDictionary<string, StackParameter> stackParams)?
I think this would be really useful for those who prefer to cdk synth the stack and obtain a template with well defined parameters and branch the stack deployment process from there without using cdk deploy

[kennu]

I would also like to see parameter support, so that AWS CDK can be used to generate CloudFormation templates for any purpose where the workflow is already based on parameters. E.g. the template is validated by a testing / approval process and parameters are then used to deploy it to multiple places.

[eladb]

I apologize that this issue was closed. We don't have an objection for supporting parameters, but just haven't prioritized this work. As mentioned above, using CloudFormation parameters is generally an anti-pattern for CDK apps given "synth-time" resolution is more deterministic and allows you to reason about values in your code, but we understand that people who come from existing CloudFormation workflows may still want to leverage parameters.

We will gladly accept a PR to that end if someone is interested in picking this up, or eventually we'll get to adding this support.

The general approach that I would take is to simply allow passing --parameters switches to cdk deploy (either in command line or through cdk.json).

[akirsman]

Actually, I was able to add parameters to the template through this:

            var bucketNameParam = new CfnParameter(this, "bucketName", new CfnParameterProps() { Type = "String", Description = "bucket name" });
            SetParameterValue(bucketNameParam, this.Node.GetContext("bucketName").ToString());
            var bucket = new Bucket(this, "poc-bucket", new BucketProps() { BucketName = bucketNameParam.Value.ToString() });

This way I was able to "synth" a template and deploy from there without cdk deploy!

I guess this is supported usage, right? Thanks!

[eladb]

Of course it is supported :-), and as I said, no objection also supporting deploying through the CDK CLI as well.

[kennu]

Thanks @akirsman, it's good to know that is possible. I will keep this solution in mind for the future. Still kind of waiting for a 1.0 release before using CDK in customer projects..

[Doug-AWS]

https://docs.aws.amazon.com/cdk/latest/guide/get_secrets_manager_value.html

[michaelday008]

I'm trying to get something working similar to what @akirsman did and having some issues.

I used cdk init to create a project using typescript and have the standard bin/my-app.ts and lib/my-stack.ts.

I would like to be able to pass in a codeCommit repository ARN for my stack so it can create a pipeline for any codecommit repository. The use case is either a service catalog entry or just a re-usable template for quick lambda deployment.

Here is the relevant section of code in my stack:

const CodeCommitRepositoryARN = new cdk.CfnParameter (this, 'CodeCommitRepositoryARN', {
        type: 'String',
})

// get context value from command line
this.setParameterValue(CodeCommitRepositoryARN, this.node.getContext("CodeCommitRepositoryARN").toString() );

// refer to the existing codecommit repository (that the user will specify)
const CodecommitRepository = codecommit.Repository.fromRepositoryArn(this, 'LambdaBuildRepository', CodeCommitRepositoryARN.value.toString());

I invoke it from the command line like this:

cdk deploy -c CodeCommitRepositoryARN=arn:aws:codecommit:us-east-1:1234567890:some-lambda-function

However, it seems that the setParameterValue call is not actually setting the Parameter Value so I get this as output of the deploy command:

LambdaBuildPipelineStack failed: ValidationError: Parameters: [CodeCommitRepositoryARN] must have values
Parameters: [CodeCommitRepositoryARN] must have values

Is there something missing in the documentation or am I just trying to implement this wrong?

I had suspected that maybe I had to deal with the parameters at the app level, not the stack level, but the parameters and contexts are properties of a Stack, so that didn't seem to be the route to go.

[Doug-AWS]

If you call

cdk deploy -c CodeCommitRepositoryARN=arn:aws:codecommit:us-east-1:1234567890:some-lambda-function

You get the value of CodeCommitRepositoryARN with:

const ccrArn = this.node.getContext("CodeCommitRepositoryARN");

[eladb]

Indeed, it was dead-code that didn't really work. Did you use it for anything?

[michaelday008]

@eladb Here was our use case for this functionality:

We were creating service catalog entries using CDK to output the cloudformation code. Before deploying the service catalog entry, we have a need to test it and ensure that it does the right things when sent the right parameters.

Without the '-c' functionality to set parameters, this is impossible.

We ended up using aws cloudformation deploy instead of cdk deploy because at least parameters aren't broken in the aws cloudformation deploy command.

Although we weren't using it in the past, the fact that it was documented as a valid option caused much confusion when the documented option did not work as advertised.

It would be great if this could be fixed, because otherwise people are forced to use cdk synth to synth and then aws cloudformation deploy to test. From a workflow perspective, it makes sense to use cdk synth and cdk deploy together, but parameters need to be fixed for that to be possible.

[brettswift]

@michaelday008

This could work for you. Let context set defaults on the parameters in the template.

Ex:

    const paramBucketname = new cdk.CfnParameter(this, 'myParam', {
      default: this.node.tryGetContext('myParam'),
      type: "String",
    })

When default is set to false - ie no context found, default will not be rendered in the template. (as per cdk 0.35.0).

With that config:

cdk synth -c myParam=donkey

Parameters:
  myParam:
    Type: String
    Default: donkey
Resources:
~ 

cdk synth

Parameters:
  myParam:
    Type: String
Resources:
~ 

The only difficulty here is if that parameter is usable in CDK types. For me, I needed a Bucket, but even an IBucket would do:

s3.Bucket.fromBucketName(this, 'pipelineBucket', paramBucketname.valueAsString)

I can either use an external bucket or just create one if one isn't passed in. You choose at synth/ deploy time.

[defel]

Hello,

I would like to describe,

• what my problems with CFN Imports are and
• why I would use Parameters when possible

My Problem with CFN Import is, that the resources can't be updated, when they are used in other stacks. Maybe I get this wrong, but for example lets have the following stacks:

class LowLevelStack(core.Stack):
    @property
    def dependency_layer(self):
        return self._dependency_layer

    def __init__(self, scope: core.Construct, id: str, **kwargs) -> None:
        super().__init__(scope, id)

        self._dependency_layer = LayerVersion(self, "PythonDependencyLayer",
                                        code=Code.asset(
                                            "./src/lambda/dependency-layer"),
                                        compatible_runtimes=[Runtime.PYTHON_3_7],
                                        description="Provide the dependencies as an own layer"
                                        )

class HighLevelStack(core.Stack):
    def __init__(self, scope: Construct, id: str, config: Config, **kwargs) -> None:
        super().__init__(scope, id)
        self.dependency_layer = kwargs.pop("dependency_layer")

        with open('./src/lambda/app.py', "r") as file:
            file_content = file.read()
            appHandler = Function(
                self, "AppHandler",
                runtime=Runtime.PYTHON_3_7,
                handler="index.handler",
                code=Code.inline(file_content),
                layers=[
                    self.dependency_layer
                ],
                environment={
                    "STAGE": config.stage
                }
            )

app = core.App()
config = get_config()

lowLevelStack = LowLevelStack(
    app, f'low-level-python-{config.stage}',
    config=config
)

highLevelStack = HighLevelStack(
app, f"high-level-python-{config.stage}",
    config=config,
    dependency_layer=lowLevelStack.dependency_layer
)

(Explanation: We have a LowLevelStack providing a Lambda-Layer Resource and a HighLevelStack which uses the lambda-layer to define a Lambda-Function)

When I deploy this app, everything works and is fine.

But at a later moment, when I refactor this - for example when I move the LambdaLayer from the LowLevelStack to an other Stack, I get the following error from CloudFormation:

Export low-level-python-dev:ExportsOutputRefPythonDependencyLayer12345BEEF... cannot be updated as it is in use by high-level-python-dev

This message is absolute correct and I can do nothing to correct this. I have to delete everything and deploy from scratch. This might be ok or not, depends on which resources are additionally defined in the stack (classic example for me is S3-Bucket when I have to manually delete the resource - or even better a CloudFront Distribution .. lunch time). What I really want is: Update resources in low-level stacks, without the need to delete the low-level stacks. (On a side note: nested stacks are even worse in this use case)

I think the root-reason for this is: Cloudformation handles the dependencies between the stacks when I use Fn:Import.

And this is why I never ever use Fn:Import in my Cloudformation-Templates - too often it ends in a state where I have to delete everything and start over from beginning.

In my ideal world, CDK would use CFN Parameters and handles the dependency between the stacks by itself and delegates the cross-stack values to CFN parameters.

The process for my use-case above would look like this:

• CDK creates a dependency graph of the stacks and update the stacks in this order (this is already done? )
• First the low-level stack get updated. The LambdaLayer resource is removed from this stack.
  • This is no problem for the lambda function in the high-level stack, the Lambda-Function will still work, I tested this.
• The new stack with the LambdaLayer gets deployed and defines it Outputs
• The HighLevel Stack gets updated, with the new resources passed to parameters

One tool I used before CDK was Sceptre which handles this parameter/dependency stuff very well.

Just my input to the question where parameters may be useful.

p.s.: I can provide the example above in Kotlin or Typescript and can setup a test-repo if required.

p.p.s: Maybe I structure my stacks wrong? How do you structure your stacks? Do you also get the .. cannot be updated as it is in use by .. - error from time to time?

[defel]

Ok, it happened again - this time with ECS-Cluster lowlevel and ECS-Service hihglevel: AutoScalingGroup (defined in my ECS-Cluster construct) cannot be updated, as it is used in the highlevel stack.

How should I understand the model behind this?

As far as I understand it for now is:

Everytime I share resources between stacks, these resources should never get an update (or have a retain-policy). All dependencies are hard dependencies. When there is an update on resources, which have dependencies to other stacks, I have to delete the whole other stack(s) which have a dependency on this resource - so I can update/replace this single resource.

In my case this means that I have to backup the rds, recreate the kms secrets, etc.

I feel that this should not be such a yak-shaving everytime, but it happends even when there are just little updates.

I talked about this topic in the og-aws slack, and @ryansb pointed out to use SSM Parameter Store for this as he documented this here: https://www.trek10.com/blog/cloudformation-splitting-and-sharing/

Quick check shows that cdk supports reading from ssm, but not writing: https://docs.aws.amazon.com/cdk/latest/guide/get_ssm_value.html

I will go down this path and will update this issue as soon as I have some results on this.

Ideal solution for me is, to find a method to fade-in and fade-out resources in the stacks by myself.

Edit: see #4014 for a feature request regarding ssm parameter store

[michaelday008]

@brettswift

Thanks for that. I ended up using a slightly modified version of this which seems to be working for my use case.

const gitBranchParam = new cdk.CfnParameter(this, 'GitBranch', {
    type: 'String',
    description : 'Enter the git branch to monitor',
    default: this.node.tryGetContext('gitBranch') ? this.node.tryGetContext('gitBranch') : 'dev'
  });

That code allows me to do a simple cdk synth command which will result in a cloudformation template with dev as the default GitBranch parameter value, which is necessary for the creation of the Service Catalog entry to show users a sane default

Parameters:
  GitBranch:
    Type: String
    Default: dev

If I want I can also test a synth directly from the command line and override that parameter using

cdk synth -c gitBranch=somebranchname

The result is

Parameters:
  GitBranch:
    Type: String
    Default: somebranchname

[ghost]

I am currently working on a way to add CloudFormation parameters to cdk deploy. I am working on it under the issue #1237

[hynynen]

Hi!

I'm not sure if this is relevant to this particular case, but I ended up using CfnParameters while working with ADF (https://github.com/awslabs/aws-deployment-framework). ADF provides a way to define variable in different scopes, like global, regional, per-OU or per-account. Since ADF builds templates/apps in a special deployment account (and we are using CodeBuild) and deploys result as CloudFormation in target account, there must be a way to enter CDK parameters relevant to any individual target account. ADF parses parameters to separate parameter file and gives that as argument when deploying CloudFormation.

My first use-case is enabling flow log delivery to centralized logging account. VPC's and flow logs have been defined elsewhere at some time in history. Creating new flow (avoiding manually configuring existing ones) requires knowledge of VPC Id's in target account. I would rather enter them as parameters in ADF than start an IAM shitstorm/mapping all accounts to VPC Id's in my code.

Just a side note, new accounts will have this log shipping defined as the VPC's are defined. I just working a patch for the old accounts.

(Edited some typos)

[ghost]

@hynynen If I understand correctly, you can just define your stacks to point to different regions, accounts, you name it, and in the next version of CDK (v1.28.0) you will be able to pass deployment parameters to a given stack, by passing cdk deploy --parameters "YourStack:ParamKey=ParamValue" -- YourStack.

[hynynen]

I'm not sure if that really covers this case. And maybe I don't know how to express it properly :) I still appreciate that feature, though.

ADF team describes it better: https://github.com/awslabs/aws-deployment-framework/blob/master/docs/user-guide.md#cloudformation-parameters-and-tagging

And I want to stress that everything work for me now. There is just one clear use-case for stack parameters.

[ghost]

n.b. Parameters are documented in a new-ish topic in the CDK Developer Guide

https://docs.aws.amazon.com/cdk/latest/guide/tools.html

Should we close this?

[VarunJohar]

I face one problem with parameters for both cdk and cfn , when I update any parameter value cdk or cfn both not getting updated since it is not a change in cdk code and for re deploy my changes I first need to delete my stack and then again deploy.
Please suggest any solution for this

[ghost]

@VarunJohar Have you tried using the --force flag?

[ccfife]

Closing this issue as complete, see: https://docs.aws.amazon.com/cdk/latest/guide/parameters.html

[logemann]

Nice you can pass parameters on "cdk deploy" but why isnt it possible for "cdk synth" ? I think i can live with @michaelday008 example and do it this way, but still feels a little off.

I want to create a template via synth and process the template with a CRON based lambda via cloudformation.createStack() JS SDK. Its a bit challening because of those Cfn parameters in the template like S3Bucket or S3Key.

[eladb]

Nice you can pass parameters on "cdk deploy" but why isnt it possible for "cdk synth" ? I think i can live with @michaelday008 example and do it this way, but still feels a little off.

I want to create a template via synth and process the template with a CRON based lambda via cloudformation.createStack() JS SDK. Its a bit challening because of those Cfn parameters in the template like S3Bucket or S3Key.

@logemann Not sure I understand what you expect synth with parameters to produce. The output of synth is CFN templates.

[logemann]

Of course i know that it produces CFN templates. But it might produce templates with parameters which are w/o values. So running those templates via createStack() doesnt work. It would be nice to put in param defaults via synth command line.

Note: I am also aware of passing params via createStack().

[ghost]

You can just use the context for that. That is meant to be burned into the synthesized template, unlike parameters which are a deployment only construct.

[logemann]

Yeah... thats what @brettswift mentioned. I am aware of that. Just thought of why not just putting a -p which directly translates to parameter defaults. So basically the same what brett achieved with the code but baked right into the command line.

[ghost]

That kind of makes sense. Though I think this will make the usage of parameters between synth and deploy inconsistent. So then you could synth something with synth that you will not be able to synth through the deploy command, unless making code changes.

[logemann]

Dont know the process in detail, but in my case, the parameters i want to have defaults for are not "my" parameters but the ones created by CDK. I just want put values in there.

So basically parameters like those:

"Parameters": {
    "AssetParameters4c2988a57571fd4c34de12bae67441541aeea1a59e085f95e5b708922ff45061S3Bucket0A1029B1": {
      "Type": "String",
      "Description": "S3 bucket for asset \"4c2988a57571fd4c34de12bae67441541aeea1a59e085f95e5b708922ff45061\""
    },

[ghost]

Yeah those are usually handled by cdk at deployment time and are unrelated to the parameters the user needs to pass in. Often these are based on objects that cannot be known at synthesis time, which is why they are postponed until deployment time.
Though that is where my knowledge of those end.