StackSets Support

[eladb]

Description

Allow users to define a template through AWS CDK constructs and deploy as a Self-Managed CloudFormation StackSet.

• User defines stack set the same way as normal cdk stack
• When user executes cdk deploy it would create a stack set if it does not exist or update an existing stack set + existing stack set instances.

Proposed Scope:

• Create New Self-Managed Stack Set (docs).
• Update Existing Self-Managed Stack Set:
  • Update includes updating existing Stack Set instances as well
  • https://docs.aws.amazon.com/AWSCloudFormation/latest/APIReference/API_UpdateStackSet.html
  • https://docs.aws.amazon.com/AWSCloudFormation/latest/APIReference/API_UpdateStackInstances.html
• Ability to set stack set configuration/deployment properties:
  • Administration Role, Execution Role, Fault Tolerance, Concurrency, etc.

Out of Scope for v1 (May be iteratively added in future):

• Service-Managed Stack Set
• Add/Remove stack set instances by ID or OU

Roles

Role	User
Proposed by	@linsona
Author(s)	@linsona
API Bar Raiser	@skinny85
Stakeholders	@alias, @alias, @alias

See RFC Process for details

Workflow

• Tracking issue created (label: status/proposed)
• API bar raiser assigned (ping us at
  #aws-cdk-rfcs if needed)
• Kick off meeting
• RFC pull request submitted (label: status/review)
• Community reach out (via Slack and/or Twitter)
• API signed-off (label api-approved applied to pull request)
• Final comments period (label: status/final-comments-period)
• Approved and merged (label: status/approved)
• Execution plan submitted (label: status/planning)
• Plan approved and merged (label: status/implementing)
• Implementation complete (label: status/done)

---

Author is responsible to progress the RFC according to this checklist, and
apply the relevant labels to this issue so that the RFC table in README gets
updated.

Related: aws/aws-cdk#11896

[paulfryer]

Yes, I have global applications that I need to deploy the same infrastructure in multiple regions. Using stacksets the CF templates need to use ${AWS::Region} and ${AWS::AccountId} variables to set the right values for each region/account. Looking at the Synth'd CF templates from CDK there are no variables like this. My suggestion is to use variable everywhere possible so we can support multiple regions and accounts, and StackSets. At this point I don't think I can use CDK for global applications with StackSets because of this limitation.

[paulfryer]

I built something like CDK in the past and to get it to work with StackSets I had to copy Lambda code to a bucket in each region. This is because Lambda wants the code in the same region. I'd assume we'll need to do something similar for CDK to support StackSets.

[eladb]

still relevant

[aksdari]

Currently, I am trying to switch from troposphere to AWS CDK. The only issue that is stopping me or actually just limiting my options with AWS CDK is deployment using StackSets.

The issue is as following:

• I am creating a lambda using aws-lambda.Function and Code.asset which relies on using cdk deploy to inject the parameter values for the S3 bucket, object key and version.

I can get away if I use lower level construct CfnFunction but it defeats the purpose imo.

are there any workarounds you guys recommend? at least until it is natively supported.

[eladb]

I believe that when we implement aws/aws-cdk#3437 and asset locations will be hard-coded (without parameters), this will be enabled.

[SoManyHs]

Had a customer today ask about stackset support. Is aws/aws-cdk#3437 still the going solution?

[timpur]

Any update on this ? since aws/aws-cdk#3437 is merged ?

[AntonioAngelino]

Any news?

[apidinom]

Any update on this ?

[JamieMcKernanKaizen]

It would be good to see some progress on this

[AntonioAngelino]

@eladb Do you have any update? 🙏

We would love to use AWS CDK in combination with AWS Org. and CloudFormation StackSets.

[corrjo]

Do we know what actually needs to be done to implement this? Would love to be able to deploy multiple instances from a single place. Happy to help however I can to make this happen.

[eroteme]

I would also love to be able to deploy stacksets via CDK.
We have lots of accounts under a centrally managed organisation account. Recently our team was required to create all sorts of AWS resources in these accounts for initialisation purposes. The aws resources required would be defined via Cloudformation from other teams and applying them in the main org account using stacksets was a good way to achieve this.
We basically create the stackset in the org account (service managed) and apply stackset instances to an org tree based on OU; any accounts under that OU will get the resources required.

The issue we had is that some teams wanted to use CDK to create their Cloudformation but stacksets are not currently supported. The way we got around this was to make the a cloudformation template via cdk synth and apply that template using simple bash script using aws-cli. It wasn't a pretty solution but worked. We didn't want to put too much dev time into making lambda functions/custom resource definitions etc if this ticket was close to being complete. Do we know how long until stacksets are supported under CDK?

From what issues I saw, CDK needs to be able to do the following:

1. As we used service managed permissions we need to allow the org account access to all sub accounts ( as per https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/stacksets-orgs-enable-trusted-access.html).
2. Allow CDK to create a stackset into a single account (org account) using a cloudformation template (from 1 or more CDK apps)
3. Have a way for CDK to apply stacksets instances to (either accountId or Org OU) and region

Maybe there is a better way, I'm open to suggestions :)

[zoonderkins]

We also have 2 OU and each has its own sandbox. I hope cdk can support stacksets too~~
https://aws.amazon.com/blogs/aws/new-use-aws-cloudformation-stacksets-for-multiple-accounts-in-an-aws-organization/

[dejonghe]

Also critical for my team, anyone using Control Tower for large orgs needs this.

[eladb]

Also critical for my team, anyone using Control Tower for large orgs needs this.

@dejonghe can you please describe your use case in more detail? In what way do you wish to use stack sets?

You can technically define a CDK stack, synth a template and deploy it through StackSets. What additional support would you expect from the CDK for this scenario?

[dejonghe]

I guess that routes seems like it would fit.
I would say it would be nice to be able to use the same deploy mechanism for stack sets. We're moving off of a custom cloudformation deployment tool that prepped out CFN, stashed it in S3 in a versioned prefix, as well as many other things but we had support for stacksets.
Use case is that we are using Control tower and have things that need to be deployed to every account, and should be controlled / owned by the master account.
What I'd like to see is a core.StackSet that takes a list of accounts or an OU or something. I know that this isn't a simple change based on the way things are built up in CDK.
Using the same deployment mechanism is really what I would want.

[IngussNeilands]

👀

[dsudduth]

@eladb I would have to agree with @dejonghe.

Between CloudFormation, AWS CDK, and AWS SAM, the ProServe team definitely get a ton of questions from our customers surrounding the variations in tooling and lack of (full) integration. Control Tower and StackSets are the best option we have for easily deploying and maintaining services that need to exist within every account; a common pattern for the enterprise where operational resources are centrally managed.

CDK has the potential to be the go-to tool. I know I'm a fan. But, we definitely need to solve for "deployment at scale" whether that means integrating with StackSets or a separate mechanism. Either way, it would be great if we can avoid context switching (different tools) to get the job done.

[paul-ge]

Could below help?: -

https://docs.aws.amazon.com/cdk/latest/guide/stack_how_to_create_multiple_stacks.html

[dsudduth]

Could below help?: -

https://docs.aws.amazon.com/cdk/latest/guide/stack_how_to_create_multiple_stacks.html

Hi @paul-ge ,

I'm afraid not. These are fundamentally different ideas. The doc you shared provides an example of multiple CDK stacks which makes it easy to modularize code, but still deploys to a single account. What we're discussing is deployment across all accounts by leveraging managed StackSets in CloudFormation.

[pgarbe]

Is anyone working on that? I'd like to have a StackSet constructs which accepts a Stack property as input. But I'm running into issues during synth, as the "StackSetStack" requires the output of the other stack which will be written only at the end of the synth.

[McDoit]

I have just stumbled on this with StackSets and the CDK, but I assume one of the bigger issues would be the assets CDK provisions in S3 and lambda for Custom Resources and similar applications?
Is there any way to gather them in a stack and apply aws:PrincipalOrgPaths
https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html
to their access roles?

[jacobsnapp]

As a customer working closely with an AWS ProServ team, I am very interested in more support for StackSets with CDK. We currently manage many StackSets across multiple AWS Control Tower environments using custom tooling wrapped around CDK.

I recently used the pattern suggested by @redbaron leveraging CfnStackSet and it got me very excited! I would love to see more CDK support for StackSets, including the ability to import existing StackSets into CfnStackSet or the L2 equivalent.

[RaeesBhatti]

It is possible to deploy CDK as a StackSets by using intermediateStage. It is not pretty, but doable:

const app = new cdk.App()

// Stage allows us to synth CF template for Stacks added to it
const stage = new cdk.Stage(app, "Stage")
new StackToDeployAsStackset(stage, "XYZ")
const stackSetTemplateObj = stage.synth().stacks[0].template

const stackSetDeployer = new cdk.Stack(app, "StackSetDeployer")
new cdk.CfnStackSet(stackSetDeployer, "XYZASaStackSet", {
            ...
            templateBody: JSON.stringify(stackSetTemplateObj),
})

This works for resources that don't require assets but not for resources like Lambda which require Asset Parameters to work. Is it possible to deploy the assets along with the StackSetDeployer?

[michft-v]

Same as above. I work deploying CDK (Java version) as an enterprise stack. Having issues with conflicts from StackSet created resources and CDK created resources. Things like AWS Config recorders information, VPC ARN and Stackset deployed IAM roles end up being manually entered into CDK which is not really the point of programatically deploying your stacks.

Interested in being kept up to date with any progress.

[tvb]

It is possible to deploy CDK as a StackSets by using intermediateStage. It is not pretty, but doable:

const app = new cdk.App()

// Stage allows us to synth CF template for Stacks added to it
const stage = new cdk.Stage(app, "Stage")
new StackToDeployAsStackset(stage, "XYZ")
const stackSetTemplateObj = stage.synth().stacks[0].template

const stackSetDeployer = new cdk.Stack(app, "StackSetDeployer")
new cdk.CfnStackSet(stackSetDeployer, "XYZASaStackSet", {
            ...
            templateBody: JSON.stringify(stackSetTemplateObj),
})

This approach seems to be broken since version 1.102.0. It now tries to deploy the staged stack also resulting in unwanted extra resources being deployed.

[McDoit]

This approach seems to be broken since version 1.102.0. It now tries to deploy the staged stack also resulting in unwanted extra resources being deployed.

UPDATE:
cdk diff --all works as intended but cdk deploy --all adds the stage as a "hidden" stack

Seems to be related to aws/aws-cdk#14379
Testing around with the patterns there and it seems like --all instead of * works as a workaround!

Was this the intended behavior @otaviomacedo ?

[tvb]

Testing around with the patterns there and it seems like --all instead of * works as a workaround!

Yes, I noticed. Thank you.

[robertjan-b]

@RaeesBhatti it is possible to publish the assets in the account where you deploy the stacksets using cdk-assets:
cdk-assets publish --path cdk.out/assembly-Stage/StackSetDeployer.assets.json --verbose

You will have to make sure that the rest of the accounts can access the assets in the S3 bucket by updating the bootstrapping template.

[RaeesBhatti]

Thanks for the suggestion @robertjan-b . I'll try that

[robertjan-b]

The way to create the stackset stack looks a lot cleaner using the following method: aws/aws-cdk#11896

[brunobodson]

Thanks for the suggestion @robertjan-b . I'll try that

@RaeesBhatti, did you manage to make it work with the assets? Looking forward to your findings!

[tvb]

It seems StackSets is kinda broken in CDKv2. It tries to resolve the cdk-bootstrap parameter from the Parameter Store in my target accounts too..

Unable to fetch parameters [/cdk-bootstrap/xxxxx/version] from parameter store for this account.

[skinny85]

@tvb do you mind opening us an issue in the main CDK repository (https://github.com/aws/aws-cdk/issues/new/choose), showing your code, and the exact command you execute, and the full error that you get?

[bsweeney-gingerio]

Using StackSets to deploy CDK stacks across accounts/OUs in an organization would be helpful for customers who need to ensure certain resources are present in every account to meet compliance requirements. For large organizations, it also obviates the need to automate CDK bootstrapping every account as part of a CDK Pipelines deployment. For additional security, I believe CDK should refuse (at least warn) to bootstrap StackSets in the Organizations management account, instead requiring that the bootstrap environment exist in a StackSets delegated administrator.

In the meantime, one way my team gets StackSet support from CDK (in addition to previous suggestions) is to write the synthesized template to the CDK assets S3 bucket:

const stackTemplateAsset = new assets.Asset(
  this,
  `${stackClass.name}TemplateAsset`,
  {
    path: stackTemplatePath,
  }
);

stackTemplatePath is the filesystem path to the synthesized template written to a temporary directory, since I didn't see a way to make this work directly with the JSON. The template can then be passed to the new CfnStackSet as templateUrl: stackTemplateAsset.httpUrl.

[jabalsad]

Is this RFC still moving forward?

[pgarbe]

There seems to be some movement for ServiceCatalog Products. They support now assets.

I'm working on a similar solution for StackSets here: https://github.com/pgarbe/cdk-stackset

[PrettySolution]

Do you really need StackSets if you have waves? what is the use case?

    const subAccountLevelWave = pipeline.addWave('subAccountLevelWave')
    subAccountLevelENVs.forEach(env => {
      subAccountLevelWave.addStage(new AccountLevelStage(this, `level-${env.region}-in-${env.account}`, {
        env
      }))
    })

[bsweeney-gingerio]

Use case: ensure that any account added to an organization has a CDK bootstrap template applied so that it can be included in a pipeline wave. Manage that template using CDK.

[gbvanrenswoude]

As long as you do not use Assets in your ProductStack this works pretty neat:

export const createStackSet = (
  scope: Construct,
  name: string,
  stack: sc.ProductStack,
) => {
  return new CfnStackSet(scope, name, {
    permissionModel: 'SERVICE_MANAGED',
    autoDeployment: {
      enabled: true,
      retainStacksOnAccountRemoval: false,
    },
    capabilities: [
      'CAPABILITY_IAM',
      'CAPABILITY_NAMED_IAM',
      'CAPABILITY_AUTO_EXPAND',
    ],
    description: `StackSet for ${name}`,
    operationPreferences: {
      failureToleranceCount: 25,
      maxConcurrentCount: 25,
    },
    parameters: [],
    stackSetName: name,
    templateUrl:
      sc.CloudFormationTemplate.fromProductStack(stack).bind(scope).httpUrl,
    stackInstancesGroup: [
      {
        deploymentTargets: {
          organizationalUnitIds: [
            scope.node.tryGetContext(ContextProps.OU_ROOT_ID),
          ],
        },
        regions: ['eu-west-1'],
      },
    ],
  });
};

[ericzbeard]

See https://github.com/cdklabs/cdk-stacksets/

[RichiCoder1]

See cdklabs/cdk-stacksets

Ayy, awesome to see something semi official! I wonder if the asset work from @pgarbe's repo (https://github.com/pgarbe/cdk-stackset) couldn't be carried over to partially address cdklabs/cdk-stacksets#58

[evgenyka]

Closing this RFC in favor of https://github.com/cdklabs/cdk-stacksets/. Please move the relevant conversation to that repo.

[stekern]

Are there any plans to move the construct from https://github.com/cdklabs/cdk-stacksets/ to the main AWS CDK repository at https://github.com/aws/aws-cdk?

It's not entirely clear to me what the different governance model and lifecycle is for code in these two different GitHub organizations. To me it seems like cdklabs hosts code that is less official, more experimental and potentially has less support and development, so I tend to be vary of using it in production.

Could you provide any insights on this?