force_new_deployment does not redeploy the ecs service

[nbrys]

Terraform Version

0.12.18

Affected Resource(s)

aws_ecs_service

Terraform Configuration Files

resource "aws_ecs_service" "app_service_aws_no_lb" {
  count                = local.targetgroups == [] ? 1 : 0
  name                 = var.name
  cluster              = data.aws_ecs_cluster.cluster.arn
  task_definition      = "${aws_ecs_task_definition.app.family}:${aws_ecs_task_definition.app.revision}"
  force_new_deployment = var.force_new_deployment

  deployment_maximum_percent         = var.autoscaling["deployment_maximum_percent"]
  deployment_minimum_healthy_percent = var.autoscaling["deployment_minimum_healthy_percent"]
  desired_count                      = var.autoscaling["desired_capacity"]
  launch_type                        = var.type

  dynamic "load_balancer" {
    for_each = local.targetgroups.*
    content {
      target_group_arn = load_balancer.value.arn
      container_name   = var.main_application
      container_port   = load_balancer.value.port
    }
  }

  network_configuration {
    subnets         = var.subnets
    security_groups = var.ecs_security_group_id
  }

  dynamic "service_registries" {
    for_each = aws_service_discovery_service.service_discovery.*
    content {
      registry_arn = service_registries.value.arn
      port         = var.service_discovery_port
    }
  }

  dynamic "ordered_placement_strategy" {
    for_each = var.type == "FARGATE" ? [] : [local.ordered_placement_strategy]
    content {
      field = lookup(ordered_placement_strategy.value, "field", null)
      type  = ordered_placement_strategy.value.type
    }
  }

  dynamic "placement_constraints" {
    for_each = var.type == "FARGATE" ? [] : [local.container_placement_contstraints]
    content {
      expression = lookup(placement_constraints.value, "expression", null)
      type       = placement_constraints.value.type
    }
  }

  # Preserve desired count when upgrading on peak moments
  lifecycle {
    ignore_changes = [desired_count]
  }

  propagate_tags = "SERVICE"

  tags = {
    Environment = var.environment
    Project     = var.project
    Subproject  = var.subproject
  }
}

Expected Behavior

When force_new_deployment is set to true and you perform a terraform run, the service should be redeployed. All other settings remain the same including the ECS image tag.

Actual Behavior

Terraform reports that there is nothing to redeploy and the service isn't redeployed:
No changes. Infrastructure is up-to-date.

Steps to Reproduce

1. Create an ECS service and run a terraform apply without any changes to the ECS image

[lukasbernat]

same issue here. no changes - even if new image is behind image:tag
I was under the impression that force_new_deployment redeploys the ecs task every terraform run.

[bflad]

Hi folks 👋 Terraform needs some value to show a difference for the resource update function to run. If there are no updates, the update function (and the call to force new deployment) cannot be triggered.

We may be able to work around this by introducing a new "triggers" argument, similar to the API Gateway deployment resources, to allow operators to provide their own redeployment criteria outside a lack of ECS service configuration changes.

[Kevin-Molina]

Running into the same.
Desired count: 2
min %: 50
max %: 100

Expecting it to drop the service to 1 task, deploy, drop to 1 again, deploy. It results in a no-op, but works from the console.

Since the intent of force_new_deployment is pretty much for this exact situation, but @bflad - you're saying Terraform doesn't see a config change, so doesn't attempt to hit the API, effectively rendering force_new_deployment mostly useless right?

Outside of (off the top of my head) versioning the task/images, there's no way to force this at the moment right?

[bflad]

@Kevin-Molina force_new_deployment is useful for other scenarios, as noted in the resource documentation:

roll Fargate tasks onto a newer platform version, or immediately deploy ordered_placement_strategy and placement_constraints updates

Those will be updated due to platform_version or ordered_placement_strategy/placement_constraints respectively showing value updates.

[jamroks]

In this case the document is misleading because it also stated that :

This can be used to update tasks to use a newer Docker image with same image/tag combination (e.g. myimage:latest)

Maybe the documentation should be updated to remove that statement , I see lot of people being confused but this and spending time trying to debug something that obviously isn't expected to work that way

[jamroks]

@Kevin-Molina force_new_deployment is useful for other scenarios, as noted in the resource documentation:

roll Fargate tasks onto a newer platform version, or immediately deploy ordered_placement_strategy and placement_constraints updates

Those will be updated due to platform_version or ordered_placement_strategy/placement_constraints respectively showing value updates.

In this case the document is misleading because it also stated that :

This can be used to update tasks to use a newer Docker image with same image/tag combination (e.g. myimage:latest)

Maybe the documentation should be updated to remove that statement , I see lot of people being confused by this and spend/wast time trying to debug something that obviously isn't expected to work that way

[remilapeyre]

Hi folks 👋 Terraform needs some value to show a difference for the resource update function to run. If there are no updates, the update function (and the call to force new deployment) cannot be triggered.

We may be able to work around this by introducing a new "triggers" argument, similar to the API Gateway deployment resources, to allow operators to provide their own redeployment criteria outside a lack of ECS service configuration changes.

For the time being it would be possible to change the value of a tag to get this behavior but Terraform tries to be smart in resourceAwsEcsServiceUpdate() and does not update the service when just the tags changed. I think https://github.com/terraform-providers/terraform-provider-aws/blob/master/aws/resource_aws_ecs_service.go#L922-L929 could be changed to:

	conn := meta.(*AWSClient).ecsconn
	updateService := aws.Bool(d.Get("force_new_deployment").(bool))
	input := ecs.UpdateServiceInput{
		Cluster:            aws.String(d.Get("cluster").(string)),
		ForceNewDeployment: updateService,
		Service:            aws.String(d.Id()),
	}

so that users could change the value of a tag and get the service redeployed.

[dvasiljevic-humanity]

Any update or workaroundon this? How to force update (recreation) of task if there are no changes in the task definition or service config?

[zzFernando]

Same issue here

[amithkumarg]

As a workaround, you can use the docker image digest as part of container definition, to trick terraform to see a change in state and thus create a new task definition, which will enforce service redeploy.

data "aws_ecr_image" "app" {
  repository_name = "my-image-name"
  image_tag       = "my-image-tag"
}
#this will pull the latest digest for the given image from ECR
resource "aws_ecs_task_definition" "app" {
  family                   = "my-app-service-task"
  execution_role_arn       = "my-ecs-task-execution-role"
  network_mode             = "awsvpc"
  requires_compatibilities = ["FARGATE"]
  cpu                      = 512
  memory                   = 1024
  container_definitions    = <<DEFINITION
  [
    {
      "name": "my-app-name",
      "image": "my-image-name:my-image-tag@${data.aws_ecr_image.app.image_digest}",
...
#this inclusion of data image_digest will enforce task definition to change and thus service will redeploy.

The downside is a new task definition created with every deployment if that concerns you.

More details are available here:
https://amithkumarg.medium.com/terraform-force-redeploy-aws-ecs-fargate-for-same-docker-image-tag-2089b81f02c2

[zzFernando]

very nice @amithkumarg
this workaround really solve this issue

[kataokatsuki]

Thanks for the great solution!
That solved my issue. @amithkumarg

This might be a trivial thing, but I leave the description here so that others will not be confused.

According to the AWS guide, you can choose the image name format.(https://docs.aws.amazon.com/AmazonECR/latest/userguide/docker-pull-ecr-image.html)

It is described as below.

Pull the image using the docker pull command. The image name format should be registry/repository[:tag] to pull by tag, or registry/repository[@digest] to pull by digest.

So, when you want to pull by digest, you can write like
registry/repository@sha256:...

If you use both of tag and digest, it might case error like “not found: does not exist”.

Have a nice day!

[obataku]

opened a PR that (I think) will address this: #25840
usage:

resource "aws_ecs_service" "foo" {
  ...
  force_new_deployment = true

  triggers = {
    update = timestamp()  # force update in-place every apply
  }
}

[hessamalipour]

Hi there, any update there?

[YakDriver]

Closed by #25840

[github-actions]

This functionality has been released in v4.40.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.