aws_ecs_capacity_provider_attachment

[carlosrodf]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Relates #11409

Description

Add the ability to attach an existing aws_ecs_capacity_provider to an existing aws_ecs_cluster resource. Right now we can only do it on the aws_ecs_cluster resource definition. Creating and Deleting this attachment resource would act like this cli functionality https://docs.aws.amazon.com/cli/latest/reference/ecs/put-cluster-capacity-providers.html

New or Affected Resource(s)

• aws_ecs_capacity_provider
• aws_ecs_cluster
• aws_ecs_capacity_provider_attachment

Potential Terraform Configuration

resource "aws_ecs_capacity_provider_attachment" "attach" {
  ecs_cluster_arn = "arn:xxxxxx"
  capacity_providers = []
  default_capacity_provider_strategy = {
    ...
  }
}

References

• https://docs.aws.amazon.com/cli/latest/reference/ecs/put-cluster-capacity-providers.html
• https://www.terraform.io/docs/providers/aws/r/ecs_capacity_provider.html
• https://www.terraform.io/docs/providers/aws/r/ecs_cluster.html

[bflad]

Hi @carlosrodf 👋 Thank you for submitting this.

Can you please elaborate more on the use case that requires this second configuration method? In general, the support of two separate ways of configuring the same infrastructure can be confusing for operators and we need to make concessions with expected Terraform functionality to allow this (e.g. disabling drift detection in the ECS Cluster resource in this case). Another downside here is that unlike a few of our other "attachment" resources, only one of these would be able to be configured per ECS Cluster, which is the same as it exists today.

[carlosrodf]

@bflad what I noticed is that once created with the current methods available in terraform there is no way to modify or even delete the resource because of the name restriction. When i modify any of the attributes in the aws_ecs_capacity_provider resource i get the following error:

Error: error creating capacity provider: ClientException: The specified capacity provider already exists. To change the configuration of an existing capacity provider,
 update the capacity provider.
        status code: 400, request id: 0ff57813-91b3-476e-aef4-4f4bb7727c6b

And even if i remove all the capacity provider related stuff from my terraform code i get the same error when it attempts to delete it.
After i deactivaded the resource using the aws console i was able to delete the capacity provider from terraform.

[lukedd]

This would solve the issue I reported in #11409 - indeed see that it matches what I suggested there as a possible solution.
The existing configuration method is flawed in that it forces the aws_ecs_cluster to have an indirect dependency on the aws_autoscaling_group, when in fact the dependency should be the reverse for destruction to work.

[KevinGimbel]

I'm facing the same issues right now. I am trying to integrate a capacity provider into my code but cannot get it to work reliably.

As @lukedd mentioned the ECS cluster has an indirect dependency on the auto scaling group and updates become impossible once the capacity provider is created because of the name restriction mentioned by @carlosrodf.

I think an attachment resource would make this a lot more flexible, right now the only solution I found was using a random_pet resource with capacity provider.

resource "random_pet" "capacity_provider" {}

resource "aws_ecs_cluster" "cluster" {
  provider = aws.customer

  name               = local.ecs_cluster_name
  capacity_providers = ["default-provider-${random_pet.capacity_provider.id}"]
}

resource "aws_ecs_capacity_provider" "cluster_capacity_provider" {
  provider = aws.customer

  name = "default-provider-${random_pet.capacity_provider.id}"

  auto_scaling_group_provider {
    auto_scaling_group_arn         = module.ecs_asg.this_autoscaling_group_arn
    managed_termination_protection = "DISABLED"
  }
}

[KevinGimbel]

I spoke too soon, using a random_pet will not work.

[ronaldour]

Any update on this feature? I know that the AWS API is a constraint here because it doesn't allow to update nor delete the CP but an attachment resource would make this more flexible

[ctrongminh]

Another use case of this is: I cannot find where to associate the Capacity Provider with ECS cluster which created by aws_batch_compute_environment

Our use case is first create AWS batch compute environment with UNMANAGED type, which will automatically create the ECS cluster. Then, we create the Launch Template, AutoScaling Group. After that, go to the ECS Cluster created by batch compute, and create the Capacity Provider.
However, I cannot find a way to associate the ECS Cluster created by aws_batch_compute_environment with the resource aws_ecs_capacity_provider.

I try to use the resource aws_ecs_cluster with the ECS cluster name created by aws_batch_compute_environment, and aws_ecs_capacity_provider.resource.name but got error

Error: InvalidParameterException: The specified capacity provider strategy cannot contain a capacity provider that is not associated with the cluster. Associate the capacity provider with the cluster or specify a valid capacity provider and try again.

More info on #24615

[Mrono]

This is the show stopper right now to using capacity providers in my setups, since the ASG and the ECS cluster get created in separate modules (for multiple ec2 deployment groups within one ecs)

[alephmelo]

This would solve a few workarounds I've had to implement.

Specially on use cases where we're working with legacy clusters that weren't created using Terraform.

[pimperator]

  provisioner "local-exec" {
    command = "/usr/bin/aws ecs put-cluster-capacity-providers --cluster ${aws_ecs_cluster.this.name} --capacity-providers ${aws_ecs_capacity_provider.this.name} --default-capacity-provider-strategy capacityProvider=${aws_ecs_capacity_provider.this.name},weight=128,base=128 --region ${data.aws_region.current.name}"
    #    interpreter = ["/bin/sh"]
  }
}

This is a workaround that I used to have but: whenever something changes on the capacity provider or ecs-cluster there is no notification to the attachment itself; so it is useless

[pimperator]

Hi Guys, what is the status on this besides being open? What exactly blocks this issue?

[brikis98]

Just hit the bug described in #11409. This PR seems like the right fix, for as it is now, capacity providers are broken in Terraform, as you get an invalid dependency chain: aws_ecs_cluster -> aws_ecs_capacity_provider -> aws_autoscaling_group. This chain isn't valid, because on destroy, Terraform will try to delete aws_ecs_cluster first, but it can't, because the aws_autoscaling_group hasn't been deleted. So we need an aws_ecs_capacity_provider_attachment to use capacity providers without such a dependency chain. +1 on getting this PR merged.

[richardgavel]

I have an even worse problem, in that I create a full circular dependency because I reference aws_ecs_cluster.this.name in the user data for the launch template (as part of the cluster registration)

aws_ecs_cluster -> aws_ecs_capacity_provider -> aws_autoscaling_group -> aws_launch_template -> aws_ecs_cluster

I'm getting around it, of course, by having the name of the cluster and the reference in the launch template just both point to a local variable, but this attachement stuff would break the loop too.

[hboisgibault]

Hi,
Is there any update on this one or a temporary workaround ?

[bharti8085]

Any update on this issue?

[github-actions]

This functionality has been released in v3.74.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.