[cdk-pipelines] Stacks in stage doesn't get deployed in parallel when stage.add_application is used?

[karthikns16]

❓ Parallel Deployment to Multiple Accounts

The Question

[1]: CDK Pipeline does not deploy the a stack in two accounts in parallel

environments = self.node.try_get_context("environments")
stage = pipeline.add_stage("dev")
stage.add_application(EpsRoute53(self, f"dev01-route53", env=environments["dev01"], account_identifier="dev01"))
stage.add_application(EpsRoute53(self, f"dev02-route53", env=environments["dev02"], account_identifier="dev02"))

Note, environments["dev01"], - account, region combination

Result: The stack EpsRoute53 is getting ordered as follows

1. dev01-epsroute53.prepare
2. dev01-epsroute53.deploy
3. dev02-epsroute53.prepare
4. dev02-epsroute53.deploy

I had to change to using something like this to get it added pipeline stage actions in parallel.

environments = self.node.try_get_context("environments")
stage = pipeline.add_stage("dev")
stage.add_actions(pipelines.DeployCdkStackAction(action_role=iam.Role.from_role_arn(self, id="role-dev01",role_arn="arn:aws:iam::<account-dev01>:role/<pipeline-role-arn>"),
                                   stack_name="dev01-route53-domain-stack",
                                   template_path="<path of the template>",
                                   region="us-west-2",
                                   cloud_assembly_input=cloud_assembly_artifact,
                                   prepare_run_order=1,
                                   execute_run_order=2,
                                   cloud_formation_execution_role=iam.Role.from_role_arn(self, id="role-dev01a",
                                   role_arn="arn:aws:iam::<account-dev01>:role/cdk-XXXX-cfn-exec-role-<account-dev01>-us-west-2")
                                   )))
stage.add_actions(pipelines.DeployCdkStackAction(action_role=iam.Role.from_role_arn(self, id="role-dev01",role_arn="arn:aws:iam::<account-dev02>:role/<pipeline-role-arn>"),
                                   stack_name="dev02-route53-domain-stack",
                                   template_path="<path of the template>",
                                   region="us-west-2",
                                   cloud_assembly_input=cloud_assembly_artifact,
                                   prepare_run_order=1,
                                   execute_run_order=2,
                                   cloud_formation_execution_role=iam.Role.from_role_arn(self, id="role-dev02a",
                                   role_arn="arn:aws:iam::<account-dev02>:role/cdk-XXXX-cfn-exec-role-<account-dev02>-us-west-2")
                                   )))

Is this an expected behavior?

Environment

• CDK CLI Version: 1.60.0 (build 8e3f53a)
• Module Version: 1.60.0
• Node.js Version: v12.18.2
• OS: CentOS Linux release 7.8.2003 (Core)
• Language (Version): Python 3.7.9

Other information

[rix0rrr]

It's true, we could have used a runOrder parameter in there.

[karthikns16]

@rix0rrr , Thank you for reviewing my request. Any tentative of ETA on when we could get this feature added?

Even with run order say if I had two accounts, the action in runOrder 1 has to be successful for both the accounts for action in subsequent order to start. Instead of stopping the whole pipeline when a deployments fails in one of the many accounts, shouldn't it stop just the deployment just in that one account and push forward in the remaining accounts.

[karthikns16]

@rix0rrr , any eta on when this feature will be added to CDK?

[rix0rrr]

Instead of stopping the whole pipeline when a deployments fails in one of the many accounts, shouldn't it stop just the deployment just in that one account and push forward in the remaining accounts.

That is something we'll unfortunately never will be able to achieve using CodePipeline, just because of the way it works.

[kichik]

The workaround we ended up using is:

def _run_next_action_in_parallel(stage: Stage):
    current_run_order = stage.next_sequential_run_order(0)  # passing 0 means it doesn't advance the run oder
    stage.next_sequential_run_order(1 - current_run_order)  # send the order back to 1 so the next stage runs in parallel

 
stage.add_application(EpsRoute53(self, f"dev01-route53", env=environments["dev01"], account_identifier="dev01"))
_run_next_action_in_parallel(stage)
stage.add_application(EpsRoute53(self, f"dev02-route53", env=environments["dev02"], account_identifier="dev02"))
_run_next_action_in_parallel(stage)

It resets the run order after each application and allows them to run in parallel.

[rix0rrr]

Currently will be addressed as part of #10872

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.