Blue Green ECS Deployment

[arvindshekar07]

Hi ,
I am referencing this video from the 2018 Re:Invent .https://www.youtube.com/watch?v=01ewawuL-IY.

1. I am struggling to get and blue green deployment setup where I have Infrastructure stack with fargate and another stack with multiple microservice on which run on fargate.

2. Can some examples be provided on how to do blue green deployment with ECR as an source?
   The confusing part here is how to pass the docker image tag, target and task to the cdk pipeline or as a cloudformtation template to support Blue/Green Deployment.

3. It makes sense if the stack name defaults to the TS class name but then what is the use of stack name when its being provided in the cloudformtation service.

4. How to access other stacks without having to do an TS way of logical inheritance?

5. Are there any examples where based on aws accounts i can put a specific VPC or provision differently ? I know there is a conditional function for TS but I didnt find a good way to write condition based on aws account .

6. Is there any examples on how to append the export functionality with the parameters provided to CFT at compile time? I get an TS error when I try to an optional variable to the export name @

[rix0rrr]

Blue/Green deployment for ECS is a supported feature, but I'm not sure it's available through CloudFormation yet, and we definitely don't have support for it in the CDK as of yet. You will have to set it up by hand for now.

It makes sense if the stack name defaults to the TS class name but then what is the use of stack name when its being provided in the cloudformtation service.

It's not the class name, it's the second parameter that you pass to stack:

new MyStack(app, "ThisIsTheName");

CloudFormation orients itself around the concept of "stacks" and stacks must have a name to identify them.

How to access other stacks without having to do an TS way of logical inheritance?

Classes (which you would combine through inheritance) give you a "kind of stack". You instantiate a class to get an object, which represents a real Stack. For example:

class MyStack extends cdk.Stack {
   // ...
}

// MyStack is not deployable yet. It might take parameters that we haven't filled in.
// To make it real, let's instantiate it:
new MyStack(app, 'Stack1', { flavor: 'banana' });
new MyStack(app, 'Stack2, { flavor: 'raspberry' });

// In fact, I can even instantiate multiple CloudFormation stacks from the same class!
// But they must have different names. 

This is a long way to say, once you have an object you can access attributes of it and pass those around, potentially as input arguments to another stack. It works just the same as in a regular programming language.

Here's an example of defining an S3 bucket in one stack, and using it in another:

https://github.com/awslabs/aws-cdk/blob/master/packages/%40aws-cdk/aws-s3/test/integ.bucket-sharing.lit.ts#L42

Are there any examples where based on aws accounts i can put a specific VPC or provision differently ?

I wouldn't recommend you put ifs with literal accounts in your source code. For example, it seems better to tag your VPC and use VpcNetwork.importFromContext with the tags member. But if you must:

class MyStack extends cdk.Stack {
  constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    if (this.requireAccountId() !== '12345678') {
      throw new Error('You must deploy this in account 12345678!');
    }
  }
}

Is there any examples on how to append the export functionality with the parameters provided to CFT at compile time? I get an TS error when I try to an optional variable to the export name @

I'm sorry, I do not understand the question. Could you explain a bit more?

[arvindshekar07]

@rix0rrr thanks for your quick response.

It's not the class name, it's the second parameter that you pass to stack:

new MyStack(app, "ThisIsTheName");

In my use case I have the same stack being used for UAT and PROD . In this case what would you suggest?

• Appending the "ThisIsTheName"?
• Or pass a variable as parameter?
  -Any other solution

if (this.requireAccountId()
In the code you mentioned how do you get requireAccountId?

class MyStack extends cdk.Stack {
  constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    if (this.requireAccountId() !== '12345678') {
      throw new Error('You must deploy this in account 12345678!');
    }
  }
}

Is there any examples on how to append the export functionality with the parameters provided to CFT at compile time? I get an TS error when I try to an optional variable to the export name @

I'm sorry, I do not understand the question. Could you explain a bit more?

I am sorry for the lack of explanation. Imagine a stack for UAT and PROD in the same account .
I have one stack say InfastructureStack that has all the long term resources mentioned (eg ALB,fargate S3 RDS ). I have another stack called the ServiceStack that tell what ECR task and service that needs to be created in fargate and port shift them all the request to the new ones.
Now if InfastructureStack is been referenced by the ServiceStack how would it distinguish between UAT and PROD .
The one solution I had was to check exported resources in the parent stack and see if I can append to the end with the environment information . But was not able to do that as parameter is an optional var and export function can not take name that are optional .

I am able to do the same process via clouldformation YML definition a which is kindof a Blue Green deployment .
This is an attempt to do this via CDK .

[SoManyHs]

Related: #2056

[skinny85]

Since this is a duplicate of #2056 , let's consolidate by closing this issue and moving the discussion to #2056.

[konstantinj]

https://aws.amazon.com/about-aws/whats-new/2020/05/aws-cloudformation-now-supports-blue-green-deployments-for-amazon-ecs/

reopen or new issue?

[piradeepk]

Reopened to track support for blue-green deployments

[MrArnoldPalmer]

So looks like starting with the L2s we can add an option to TaskDefinition to enable/blue green. We can do this by outputting the hook and transform detailed in the docs here https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/blue-green.html

Some experimentation may be required to get this working properly with a nice API. It could constitute a new "pattern". I haven't seen a lot of examples of using the L1s to achieve this but it should be possible.

[MrArnoldPalmer]

The "considerations" section of the user guide has some details that look like they are going to complicate implementation.

• You cannot include updates to resources that trigger green deployments and updates to other resources in the same stack update, as discussed in Resource Updates that Trigger ECS Blue/Green Deployments.
• Parameters whose values are obfuscated by CloudFormation cannot be updated by the CodeDeploy service during a green deployment, and will lead to an error and stack update failure. These include:
  • Parameters defined with the NoEcho attribute.
  • Parameters that use dynamic references to retrieve their values from external services. For more information, see Using Dynamic References to Specify Template Values.
• Declaring output values or importing values from other stacks is not currently supported for templates defining blue/green ECS deployments.
• Importing existing resources is not currently supported for templates defining blue/green ECS deployments.

[MrArnoldPalmer]

As mentioned by @shamhub in #6605 there is a construct out there supporting this that may provide some guidance for implementation.

https://www.npmjs.com/package/@cloudcomponents/cdk-blue-green-container-deployment

[CONJAUMCGCG]

What is the status of Blue Green ECS Deployment for AWS CDK?

[tsykora-verimatrix]

+1

it would be great to see support for blue/green deployments in ECS/Fargate in CDK since it's available in CloudFormation already (since 19th May)

[MrArnoldPalmer]

We are investigating how to integrate this in CDK and provide an idiomatic experience. This is not a trivial task because of the limitations of the feature in cloudformation as mentioned above.

We are considering a number of options including custom resources and/or magically splitting out blue/green resources into their own stack and somehow hiding the limitations around export/import mechanics.

I would encourage anyone who really wants to leverage this feature now to check out third party construct libraries, e.g. https://www.npmjs.com/package/@cloudcomponents/cdk-blue-green-container-deployment

An RFC is likely required to flesh out the design of an implementation that could be included in the core library.

[jpSimkins]

I spent the last 2 weeks trying to get this to work. I did find that npm package but I really didn't want to use Lambdas for this as I don't need them when I manually set this up. So glad I found this ticket ❤️

I will use the npm package above to get this working and will refactor once this is properly supported by the CDK. I am just glad to see I wasn't incompetent in this. I thought I was just missing something but glad to see it's not me.

Looking forward to the update and support of Fargate Blue/Green deployments in the CDK.

Is there any ETA for this? (next 6 months, over a year) Just wondering as using the npm package will suffice for now, it's not ideal.

[fasatrix]

As mentioned by @shamhub in #6605 there is a construct out there supporting this that may provide some guidance for implementation.

https://www.npmjs.com/package/@cloudcomponents/cdk-blue-green-container-deployment

This construct is really shaky and with unpredictable behavior.. The author is not even following issues and request too

[jk2l]

@clareliguori @SoManyHs Since the CodeDeploy deployment group setting for ECS blue-green deployments is now supported by CloudFormation, are there any plans to continue on this?

I really doubt CDK can natively support this properly. the CloudFormation implementation for Blue/Green deployment have lot of restriction. One of the big issue I have with it is that the CloudFormation will fail at transform if you modify any resources that is not part of Blue/Green resources.

e.g. I have one CloudFormation with ECS service, Blue/Green, IAM such as ECS task role and execution role. Sound normally right? so what if one day I think I need to expand the ECS task role permission to allow to perform certain task such as SQS? now the CloudFormation will immediately fail to update the stack because you can't update any resources that is not also part of the Blue/Green.

The fundamental design of the whole blue/green implementation is purely wrong and it doesn't seem like it should be used in current state.

[saishandilya]

@peterwoodworth - Thanks for bringing up my issue here.
Let me give an overview on the issue I am facing, I am currently trying to implement blue/green deployment for ECS.
I followed the below articles:

Article 1 : https://aws.amazon.com/about-aws/whats-new/2020/05/aws-cloudformation-now-supports-blue-green-deployments-for-amazon-ecs/

Article 2 : https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/blue-green.html

Article 3 : https://github.com/aws-samples/aws-reinvent-2019-trivia-game/blob/master/trivia-backend/infra/cdk/ecs-service-blue-green.ts

1. I am trying to implement the architecture in Article 2 using CDK . I got a reference for implementation from Article 3, When I am trying to synthesis the template it is creating but during the deployment I am getting below error:

Error:
Resource handler returned message: "Invalid request provided: CreateService error: NetworkConfiguration must be null. (Service: AmazonECS; Status Code: 400; Error Code: InvalidParameterException; Request ID: f3d3b359-32f3-4dad-bf56-abd2552af518; Proxy: null)" (RequestToken: 73935952-e726-0020-a399-c6518da2c124, HandlerErrorCode: InvalidRequest)

Here I am using "FargetService" Construct, I am not passing any NetworkConfiguration , but CDK is assuming by itself while creating the template and I had a confirmation from CDK support team in AWS, even they confirmed that the creation of NetworkConfiguration inside the "ECS Service" was initiated during CDK Synth. When I deploy the Template using "CDK Deploy", I am facing the above error.

2. I tried to use "cfnService" this time and I am facing another Error:

Error :
Resource handler returned message: "Invalid request provided: CreateService error: LaunchType must be blank. (Service: AmazonECS; Status Code: 400; Error Code: InvalidParameterException; Request ID: 0e1f6d38-d0e8-4438-af67-68e0bf5caa48; Proxy: null)" (RequestToken: 3aab09c2-6cfa-3674-8ea0-300d2be1c4e3, HandlerErrorCode: InvalidRequest)

I am working for Farget Service, If I don't pass the LaunchType it is assuming the launch type as "EC2" which is irrevelant for Fargate Service and If I pass the launchType the stack fails returning the above error.

Implementations:

Implementation 1 :

const Service = new FargateService(this, "Service", {
cluster: cluster,
taskDefinition: taskDefinition,
healthCheckGracePeriod: cdk.Duration.seconds(10),
minHealthyPercent: 50,
maxHealthyPercent: 200,
// assignPublicIp: false,
desiredCount: props.desiredContainersCount,
deploymentController: {
type: DeploymentControllerType.EXTERNAL
},
// vpcSubnets: selection,
// securityGroup: serviceSG,
});

I tried to remove the assignPublicIp, vpcSubnets, securityGroup and see if that is causing the NetworkConfiguration in the Service but the result is still the same.

Implementation 2 :

const Service = new CfnService(this, 'Service', {
cluster: cluster.clusterName,
desiredCount: 3,
launchType: "FARGATE",
deploymentController: { type: DeploymentControllerType.EXTERNAL },
propagateTags: PropagatedTagSource.SERVICE,
});

can someone help me with this?

[dillon-odonovan]

@saishandilya

You would specify the launchType on the TaskDefinition and TaskSet, not on the ECS service.
You do need to use the CfnService construct as well instead of the more specific FargateService for the same reason (launchType for the ECS Service must be blank as mentioned in the error message).

See the highlighted portions from the trivia game sample you linked:

• https://github.com/aws-samples/aws-reinvent-2019-trivia-game/blob/eeb460b54b083184d6f873fa261fe2258a888079/trivia-backend/infra/cdk/ecs-service-blue-green.ts#L149-L154
• https://github.com/aws-samples/aws-reinvent-2019-trivia-game/blob/eeb460b54b083184d6f873fa261fe2258a888079/trivia-backend/infra/cdk/ecs-service-blue-green.ts#L167-L172

[saishandilya]

@saishandilya

You would specify the launchType on the TaskDefinition and TaskSet, not on the ECS service. You do need to use the CfnService construct as well instead of the more specific FargateService for the same reason (launchType for the ECS Service must be blank as mentioned in the error message).

See the highlighted portions from the trivia game sample you linked:

• https://github.com/aws-samples/aws-reinvent-2019-trivia-game/blob/eeb460b54b083184d6f873fa261fe2258a888079/trivia-backend/infra/cdk/ecs-service-blue-green.ts#L149-L154
• https://github.com/aws-samples/aws-reinvent-2019-trivia-game/blob/eeb460b54b083184d6f873fa261fe2258a888079/trivia-backend/infra/cdk/ecs-service-blue-green.ts#L167-L172

@dillon-odonovan - I think you might have misunderstood my question.

1. Firstly, I tried to use L2 Constructs which is "FargateService" to create FargateService as part of B/G ECS Deployment.
   const Service = new FargateService(this, "Service", {
   cluster: cluster,
   taskDefinition: taskDefinition,
   healthCheckGracePeriod: cdk.Duration.seconds(10),
   minHealthyPercent: 50,
   maxHealthyPercent: 200,
   desiredCount: props.desiredContainersCount,
   deploymentController: {
   type: DeploymentControllerType.EXTERNAL
   },
   });

The above mentioned is my implementation which while deployment throws an error below :

Resource handler returned message: "Invalid request provided: CreateService error: NetworkConfiguration must be null. (Service: AmazonECS; Status Code: 400; Error Code: InvalidParameterException; Request ID: f3d3b359-32f3-4dad-bf56-abd2552af518; Proxy: null)" (RequestToken: 73935952-e726-0020-a399-c6518da2c124, HandlerErrorCode: InvalidRequest)

Here I am not passing any value related to NetworkConfiguration but it is somehow initiating NetworkConfiguration inside "Service" Resource of ECS Template and even the CDK Support with AWS has tried to replicate and they also faced the same issue. This initiation of "NetworkConfiguration" inside the Template is from CDK which is creating the error during the deployment.

Please let me know if there is any way to stop "NetworkConfiguration" initiation as part of the "Service" resource of the ECS Template during CDK Synth.

2. As the first way of implementation failed with errors, I tried to replace the L2 Construct with the L1 i.e.., "cfnService" hoping this way might work.
   const Service = new CfnService(this, 'Service', {
   cluster: cluster.clusterName,
   desiredCount: 3,
   deploymentController: { type: DeploymentControllerType.EXTERNAL },
   propagateTags: PropagatedTagSource.SERVICE,
   });

When I used the above construct it was creating "Service" with "EC2" as the "launchType" whereas I am looking for "Fargate" based Service. It is taking the defaults which is EC2.
So , I tried passing "launchType" constructprop over here with "FARGATE" as below:

const Service = new CfnService(this, 'Service', {
  cluster: cluster.clusterName,
  desiredCount: 3,
  launchType: "FARGATE",
  deploymentController: { type: DeploymentControllerType.EXTERNAL },
  propagateTags: PropagatedTagSource.SERVICE,
});   

The above-mentioned implementation which while deployment throws an error below :

Resource handler returned message: "Invalid request provided: CreateService error: LaunchType must be blank. (Service: AmazonECS; Status Code: 400; Error Code: InvalidParameterException; Request ID: 0e1f6d38-d0e8-4438-af67-68e0bf5caa48; Proxy: null)" (RequestToken: 3aab09c2-6cfa-3674-8ea0-300d2be1c4e3, HandlerErrorCode: InvalidRequest)

If I remove launchType it is creating Service with "EC2" and if I add the launchType it is throwing the above error during deployment.

I am basically looking for the L2 construct-based implementation, so if there is anything we can do with the first case please help me with that. If that is not possible then please help me in finding a solution with the second use case using L1.

I am also attaching my code in the attach file section please have a look at that. I hope this clears what I am looking for.
blue-green-using-ecs-stack.txt

[dillon-odonovan]

You can't specify the launchType on the CfnService, it needs to be set on the TaskSet. If you follow the AWS sample closely you should not have issues. Currently you can't use an L2 construct for the service, you have to use the L1 CfnService. Your service will deploy on Fargate if you TaskSet and Blue Green hook are configured correctly despite not specifying Fargate as the launchType on the L1/L2 service construct. If it attempts to deploy on EC2 then you have an issue elsewhere (most likely in the Blue Green hook with the logical IDs).

Your implementation 2 looks pretty close, just remove the launchType from that construct. It also appears like you're not properly referencing the ECS Service logical ID in the Blue/Green hook - it looks like you're providing the name of the service. Try changing those two things and see if it works for you.

[saishandilya]

You can't specify the launchType on the CfnService, it needs to be set on the TaskSet. If you follow the AWS sample closely you should not have issues. Currently you can't use an L2 construct for the service, you have to use the L1 CfnService. Your service will deploy on Fargate if you TaskSet and Blue Green hook are configured correctly despite not specifying Fargate as the launchType on the L1/L2 service construct. If it attempts to deploy on EC2 then you have an issue elsewhere (most likely in the Blue Green hook with the logical IDs).

Your implementation 2 looks pretty close, just remove the launchType from that construct. It also appears like you're not properly referencing the ECS Service logical ID in the Blue/Green hook - it looks like you're providing the name of the service. Try changing those two things and see if it works for you.

@dillon-odonovan - Thanks for the support, I have made the respective changes it is working fine now.

[NiccoloOlivieriAchille]

Hi all,

I've been working on this for weeks, but I continue to encounter problems.
Right now my code is more or less this:

    new CfnCodeDeployBlueGreenHook(this.stack, 'CodeDeployBlueGreenHook', {
      trafficRoutingConfig: {
        type: CfnTrafficRoutingType.TIME_BASED_CANARY,
        timeBasedCanary: {
          // Shift 20% of prod traffic, then wait 15 minutes
          stepPercentage: 20,
          bakeTimeMins: 15,
        },
      },
      additionalOptions: {
        terminationWaitTimeInMinutes: 30,
      },
      // TODO: usefull to invoke a function after test traffic is routed, but before prod traffic
      lifecycleEventHooks: {},
      serviceRole: getCodeDeployRoleName(this.stage), // retrieves the correct role
      applications: [
        {
          target: {
            type: service.cfnResourceType,
            logicalId: this.stack.getLogicalId(service),
          },
          ecsAttributes: {
            taskDefinitions: [taskDefLogicalId, `${taskDefLogicalId}Green`],
            taskSets: [taskSetLogicalId, `${taskSetLogicalId}Green`],
            trafficRouting: {
              prodTrafficRoute: {
                type: CfnListener.CFN_RESOURCE_TYPE_NAME,
                logicalId: this.stack.getLogicalId(listener4prod.node.defaultChild as CfnListener),
              },
              testTrafficRoute: {
                type: CfnListener.CFN_RESOURCE_TYPE_NAME,
                logicalId: this.stack.getLogicalId(listener4test.node.defaultChild as CfnListener),
              },
              targetGroups: [
                this.stack.getLogicalId(blueTargetGroup.node.defaultChild as CfnTargetGroup),
                this.stack.getLogicalId(greenTargetGroup.node.defaultChild as CfnTargetGroup),
              ],
            },
          },
        },
      ],
    });

Given that it deploys perfectly the first time, as soon as I update anything inside the application code, this hook throws an error:
Failed to create ChangeSet cdk-deploy-change-set on RedisLoggerStack: FAILED, Transform AWS::CodeDeployBlueGreen failed with: Failed to transform template. Failed to parse the "Hooks" section for the provided template. Please ensure the AWS::CodeDeploy::BlueGreen Hook matches the expected schema.

The produced JSON is:

"Hooks": {
    "CodeDeployBlueGreenHook": {
      "Type": "AWS::CodeDeploy::BlueGreen",
      "Properties": {
        "ServiceRole": "becky-code-deploy-test",
        "Applications": [
          {
            "Target": {
              "Type": "AWS::ECS::Service",
              "LogicalID": "RedisLoggerService"
            },
            "ECSAttributes": {
              "TaskDefinitions": [
                "RedisLoggerTaskDefinition73152CDA",
                "RedisLoggerTaskDefinition73152CDAGreen"
              ],
              "TaskSets": [
                "TaskSet",
                "TaskSetGreen"
              ],
              "TrafficRouting": {
                "ProdTrafficRoute": {
                  "Type": "AWS::ElasticLoadBalancingV2::Listener",
                  "LogicalID": "BeckyApplicationLoadBalancerListenerForProduction4293A970"
                },
                "TestTrafficRoute": {
                  "Type": "AWS::ElasticLoadBalancingV2::Listener",
                  "LogicalID": "BeckyApplicationLoadBalancerListenerForTest9DD6276D"
                },
                "TargetGroups": [
                  "ServiceTargetGroupBlue9182D76F",
                  "ServiceTargetGroupGreen10661C6D"
                ]
              }
            }
          }
        ],
        "TrafficRoutingConfig": {
          "Type": "TimeBasedCanary",
          "TimeBasedCanary": {
            "StepPercentage": 20,
            "BakeTimeMins": 15
          }
        },
        "AdditionalOptions": {
          "TerminationWaitTimeInMinutes": 30
        }
      }
    }
  }

TBH I don't know where the problem can be: every logicalId reference something real (even TaskSet), except from the Green TaskDefinition and TaskSet, right?

What else should I look for??

Thanks a lot!

[cplee]

I've created a new PR #22081 that adds support for defining CodeDeploy DeploymentGroups for ECS Services. Working on a second PR that will add support for creating new Deployments through CDK directly.

EDIT: second PR is created: #22455

[RichardChester]

Just wanted to check I've understood right will we be able to describe BlueGreen CodeDeploy deployments using CDK, to the same extent as we can in cloud formation (I.E with target groups and services) once PR #22295 get merged in ?

[clareliguori]

@RichardChester Correct. You can use the L1 construct today in CDK (working example here), but PR #22295 will make it easier through an L2 construct.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[clareliguori]

The new L2 construct is now available in version 2.50.0. You can see an example here:
https://github.com/aws-samples/aws-reinvent-trivia-game/blob/master/trivia-backend/infra/codedeploy-blue-green/deployment-setup.ts#L123-L144

[dsdavis4]

@clareliguori We have followed the example you posted and when we change the tag on a task definition, we see that the current revision becomes inactive. However, we do not see any revisions created in ECS console. If we create a revision manually in ECS console, we see the code deploy project begin the blue/green deployment.

Do you know what we might be missing? Is this possible without codepipeline?

Note: Using c#

[clareliguori]

@clareliguori We have followed the example you posted and when we change the tag on a task definition, we see that the current revision becomes inactive. However, we do not see any revisions created in ECS console. If we create a revision manually in ECS console, we see the code deploy project begin the blue/green deployment.

Do you know what we might be missing? Is this possible without codepipeline?

Note: Using c#

@cplee have you seen this behavior? I haven't had an opportunity to try to reproduce it.

[cplee]

@dsdavis4 - how are you trying to trigger new CodeDeploy deployments outside of CodePipeline? Are you using the AWS::CodeDeploy::BlueGreen hook to trigger deployments?

[clareliguori]

I'm able to partially reproduce this, except I do see a new revision created while the old one is marked as inactive.

Steps with my example:

1. Deploy the example, setting up the initial stacks with the initial task definition. I see task def revision 28 as active in ECS:

$ aws ecs list-task-definitions --family trivia-backend
{
    "taskDefinitionArns": [
        "arn:aws:ecs:us-west-2:123456789012:task-definition/trivia-backend:28"
    ]
}

2. Create a new Docker image tag in the ECR repo and update the CDK code to point to that new image:

$ docker tag reinvent-trivia-backend 123456789012.dkr.ecr.us-west-2.amazonaws.com/reinvent-trivia-backend:v2
$ docker push 123456789012.dkr.ecr.us-west-2.amazonaws.com/reinvent-trivia-backend:v2
$ git diff
diff --git a/trivia-backend/infra/codedeploy-blue-green/deployment-setup.ts b/trivia-backend/infra/codedeploy-blue-green/deployment-setup.ts
index 645983e..f93b6ff 100644
--- a/trivia-backend/infra/codedeploy-blue-green/deployment-setup.ts
+++ b/trivia-backend/infra/codedeploy-blue-green/deployment-setup.ts
@@ -97,7 +97,7 @@ class TriviaDeploymentResourcesStack extends Stack {
     });
     taskDefinition.addContainer('Container', {
       containerName: 'web',
-      image: ecs.ContainerImage.fromEcrRepository(repo, 'latest'),
+      image: ecs.ContainerImage.fromEcrRepository(repo, 'v2'),
       portMappings: [{
         protocol: ecs.Protocol.TCP,
         containerPort: 80,

3. Re-deploy the CDK stack containing the task definition to create a new task definition revision:

$ npm run build
$ cdk deploy --app 'node deployment-setup.js' --require-approval never TriviaDeploymentResourcesTest

4. The old task definition revision 28 is now marked as inactive, and the new task definition revision 29 is now active.

$ aws ecs list-task-definitions --family trivia-backend
{
    "taskDefinitionArns": [
        "arn:aws:ecs:us-west-2:123456789012:task-definition/trivia-backend:29"
    ]
}
$ aws ecs list-task-definitions --family trivia-backend --status INACTIVE --max-items 1 --sort DESC
{
    "taskDefinitionArns": [
        "arn:aws:ecs:us-west-2:123456789012:task-definition/trivia-backend:28"
    ]
}

Marking the previous task definition revision as inactive is typical CloudFormation behavior, as it assumes that you will immediately update an ECS service in the same stack with the new active revision. In this case though, because the service will be updated with the new revision outside of CloudFormation with CodeDeploy, I think we need the previous revision to stay active. For example, if CodeDeploy needs to do a rollback deployment to the ECS service (attempting to re-deploy a previous revision), it will not be able to if the previous revision is inactive.

In order to keep the previous task def revision active, an UpdateReplacePolicy of "Retain" needs to be set on the task definition resource. I have updated my example with this code:

    const cfnTaskDef = taskDefinition.node.defaultChild as ecs.CfnTaskDefinition;
    cfnTaskDef.applyRemovalPolicy(RemovalPolicy.RETAIN, { applyToUpdateReplacePolicy: true });

[dsdavis4]

@clareliguori This is the behavior we are seeing as well. It's possible we might have had a misunderstanding of the expected outcome. We expected the cdk deploy to set create a new task revision with the new image and deploy it, therefore automatically kicking off the code deploy build. We see now that you are manually kicking off the ecs deploy using the ecs cli, is it possible to have this all done via cdk deploy ?

@cplee We are not using code pipeline, we are using the L2 construct EcsDeploymentGroup. But it seems we might have had a misunderstanding as to what cdk deploy would do.

[clareliguori]

Correct, the construct added for this issue does not automatically start a new CodeDeploy deployment, it only sets up the CodeDeploy deployment group resource. @cplee's PR #22455 covers kicking off deployments with cdk deploy, or you can use a CloudFormation blue-green hook instead of using CodeDeploy deployment groups. Example here.

[cplee]

For those looking for a way to start CodeDeploy deployments for ECS services within CDK, take a look at the new construct on Construct Hub to provide this functionality:

@cdklabs/cdk-ecs-codedeploy

declare const deploymentGroup: codeDeploy.IEcsDeploymentGroup;
declare const taskDefinition: ecs.ITaskDefinition;

new EcsDeployment({
  deploymentGroup,
  targetService: {
    taskDefinition,
    containerName: 'mycontainer',
    containerPort: 80,
  },
});

[dsdavis4]

@cplee Do you know if there are any plans to add this to the official cdk library?

[cplee]

@dsdavis4 - PR #22455 attempted to add EcsDeployment to the CDK but the guidance was to use the CDK construct hub as the mechanism to add this new capability.