Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aws_autoscaling_group bug - fail to set OnDemandBaseCapacity value to zero #7368

Closed
roylauer opened this issue Jan 28, 2019 · 14 comments · Fixed by #7821 or #13623
Closed

aws_autoscaling_group bug - fail to set OnDemandBaseCapacity value to zero #7368

roylauer opened this issue Jan 28, 2019 · 14 comments · Fixed by #7821 or #13623
Assignees
Labels
bug Addresses a defect in current functionality. service/autoscaling Issues and PRs that pertain to the autoscaling service.
Milestone

Comments

@roylauer
Copy link

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.11.10

  • provider.aws v1.57.0
  • provider.template v2.0.0

Affected Resource(s)

  • aws_autoscaling_group

Terraform Configuration Files

provider "aws" {
  region  = "us-west-1"
  version = "~> 1.51"
}


resource "aws_launch_template" "lt" {
  name_prefix = "LT-spot-test"
  image_id = "${var.image_id}"
  instance_type = "t2.large"
  key_name = "${var.key_name}"

  vpc_security_group_ids = [
    "${var.security_groups}",
  ]

  iam_instance_profile {
    arn = "${var.iam_instance_profile}"
  }

  lifecycle {
    create_before_destroy = true
  }
}

resource "aws_autoscaling_group" "asg" {
  name_prefix      = "ASG-spot-test"
  max_size         = "${var.size}"
  min_size         = "${var.size}"
  desired_capacity = "${var.size}"
  target_group_arns = []
  health_check_type = "EC2"
  wait_for_capacity_timeout = "2m"   # 6 minutes is the default
  wait_for_elb_capacity = "${var.size > 0 ? var.size : 0}"
  health_check_grace_period = "300"

  vpc_zone_identifier = [
    "${var.vpc_zone_identifier}"
  ]

  mixed_instances_policy {
    instances_distribution {
      on_demand_base_capacity                  = "${var.size}"
      on_demand_percentage_above_base_capacity = 0
    }

    launch_template {
      launch_template_specification {
        launch_template_id = "${aws_launch_template.lt.id}"
        version            = "${aws_launch_template.lt.latest_version}"
      }

      override {
        instance_type = "t2.large"
      }

      override {
        instance_type = "t2.xlarge"
      }
    }
  }

  lifecycle {
    create_before_destroy = true
  }
}

Debug Output

Panic Output

Expected Behavior

When setting ASG desired_capacity, max_size, min_size and on_demand_base_capacity from "2" to "0" I would expect this work.

Actual Behavior

Got the following output & error:

aws_autoscaling_group.asg: Modifying... (ID: ASG-spot-test20190128172151691300000003)
desired_capacity: "2" => "0"
max_size: "2" => "0"
min_size: "2" => "0"
mixed_instances_policy.0.instances_distribution.0.on_demand_base_capacity: "2" => "0"
wait_for_elb_capacity: "2" => "0"

Error: Error applying plan:

1 error(s) occurred:

  • aws_autoscaling_group.asg: 1 error(s) occurred:

  • aws_autoscaling_group.asg: Error updating Autoscaling group: ValidationError: Max bound, 0, must be greater than or equal to OnDemandBaseCapacity, 2.
    status code: 400, request id: aa94c480-2321-11e9-b6ea-799c575c4fd5

Steps to Reproduce

  1. terraform apply -var size=2
  2. terraform apply -var size=0

Important Factoids

References

  • #0000
@nywilken nywilken added bug Addresses a defect in current functionality. service/autoscaling Issues and PRs that pertain to the autoscaling service. labels Jan 29, 2019
bflad added a commit that referenced this issue Mar 6, 2019
…ance_distribution on_demand_base_capacity and spot_max_price argument zero values

References:

* #7368
* https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_InstancesDistribution.html
* https://github.com/aws/aws-sdk-go/releases/tag/v1.17.5

The `mixed_instances_policy` configuration block `instance_distribution` configuration block `on_demand_base_capacity` argument was erroneously omitted from the API requests when configured to 0. This has been fixed.

The `mixed_instances_policy` configuration block `instance_distribution` configuration block `spot_max_price` argument now allows unconfiguring the value via an empty string as of AWS Go SDK 1.17.5. This has been added and verified.

Output from acceptance testing:

```
--- PASS: TestAccAWSAutoScalingGroup_ALB_TargetGroups (208.32s)
--- PASS: TestAccAWSAutoScalingGroup_ALB_TargetGroups_ELBCapacity (380.94s)
--- PASS: TestAccAWSAutoScalingGroup_autoGeneratedName (57.41s)
--- PASS: TestAccAWSAutoScalingGroup_basic (277.63s)
--- PASS: TestAccAWSAutoScalingGroup_classicVpcZoneIdentifier (79.47s)
--- PASS: TestAccAWSAutoScalingGroup_emptyAvailabilityZones (58.64s)
--- PASS: TestAccAWSAutoScalingGroup_enablingMetrics (172.07s)
--- PASS: TestAccAWSAutoScalingGroup_importBasic (237.78s)
--- PASS: TestAccAWSAutoScalingGroup_initialLifecycleHook (694.88s)
--- PASS: TestAccAWSAutoScalingGroup_launchTemplate (44.01s)
--- PASS: TestAccAWSAutoScalingGroup_LaunchTemplate_IAMInstanceProfile (52.89s)
--- PASS: TestAccAWSAutoScalingGroup_launchTemplate_update (142.34s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy (80.87s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_InstancesDistribution_OnDemandAllocationStrategy (83.04s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_InstancesDistribution_OnDemandBaseCapacity (113.14s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_InstancesDistribution_OnDemandPercentageAboveBaseCapacity (80.93s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_InstancesDistribution_SpotAllocationStrategy (141.63s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_InstancesDistribution_SpotInstancePools (114.38s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_InstancesDistribution_SpotMaxPrice (76.59s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_LaunchTemplate_LaunchTemplateSpecification_LaunchTemplateName (49.45s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_LaunchTemplate_LaunchTemplateSpecification_Version (80.60s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_LaunchTemplate_Override_InstanceType (78.31s)
--- PASS: TestAccAWSAutoScalingGroup_namePrefix (56.71s)
--- PASS: TestAccAWSAutoScalingGroup_serviceLinkedRoleARN (82.65s)
--- PASS: TestAccAWSAutoScalingGroup_suspendingProcesses (211.74s)
--- PASS: TestAccAWSAutoScalingGroup_tags (347.60s)
--- PASS: TestAccAWSAutoScalingGroup_terminationPolicies (75.88s)
--- PASS: TestAccAWSAutoScalingGroup_VpcUpdates (77.10s)
--- PASS: TestAccAWSAutoScalingGroup_WithLoadBalancer (383.31s)
--- PASS: TestAccAWSAutoScalingGroup_withMetrics (55.06s)
--- PASS: TestAccAWSAutoScalingGroup_withPlacementGroup (142.28s)
```
@bflad
Copy link
Contributor

bflad commented Mar 6, 2019

Pull request submitted: #7821

@bflad bflad added this to the v2.1.0 milestone Mar 6, 2019
@bflad
Copy link
Contributor

bflad commented Mar 6, 2019

The fix for this has been merged and will release with version 2.1.0 of the Terraform AWS Provider, likely in the next day or so.

@bflad
Copy link
Contributor

bflad commented Mar 8, 2019

This has been released in version 2.1.0 of the AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

@roylauer
Copy link
Author

Hi @bflad,

I tested this in latest version (v2.2.0) and still have the same issue in the following scenario:

  1. terraform apply -var size=1 -var image_id=ami-093bd10d3d9013341
  2. terraform apply -var size=0 -var image_id=ami-0330ce7ce2f6ae44c

aws_autoscaling_group.asg: Modifying... (ID: ASG-spot-test20190320123311924800000003)
desired_capacity: "1" => "0"
max_size: "1" => "0"
min_size: "1" => "0"
mixed_instances_policy.0.launch_template.0.launch_template_specification.0.version: "5" => "6"
wait_for_elb_capacity: "1" => "0"

Error: Error applying plan:

1 error(s) occurred:

  • aws_autoscaling_group.asg: 1 error(s) occurred:

  • aws_autoscaling_group.asg: Error updating Autoscaling group: ValidationError: Max bound, 0, must be greater than or equal to OnDemandBaseCapacity, 1.
    status code: 400, request id: 3ffb09f6-4b0f-11e9-be5a-2dff37296df3

I noticed that the issue happens when:
max_size is to be changed from 1 to 0 and at the same time mixed_instances_policy.0.launch_template.0.launch_template_specification.0.version: is being changed as well (e.g. as a result of AMI ID change in the Launch Template.

Can you please re-open the issue?

Thanks,
Roy

@bflad bflad reopened this Mar 20, 2019
@bflad
Copy link
Contributor

bflad commented Mar 20, 2019

Is your configuration different now than before? In the original report its showing this in the plan:

mixed_instances_policy.0.instances_distribution.0.on_demand_base_capacity: "2" => "0"

Which wouldn't have previously updated correctly before version 2.1.0 of the Terraform AWS Provider. Now however, your new output is not showing that update. Your mixed_instances_policy configuration block instances_distribution configuration block on_demand_base_capacity argument always needs to be less than or equal to the max_size according to the AutoScaling API rules. The updates to both max_size and mixed_instances_policy occur in the same UpdateAutoScalingGroup API call (at least in the case where the Error updating Autoscaling group error messaging is triggered) so this appears to be more of a configuration issue.

@bflad bflad added the waiting-response Maintainers are waiting on response from community or contributor. label Mar 20, 2019
@roylauer
Copy link
Author

The original scenario was indeed resolved in 2.1.0.

However now it is still failing in another specific scenario where on_demand_base_capacity is originally equal to 1 (as well as the max_size) and both being changed to zero and at the same time the Launch Template version is also being changed.

This failure does not happen if the value of on_demand_base_capacity and max_size are being changed from 2 to zero.

@ghost ghost removed the waiting-response Maintainers are waiting on response from community or contributor. label Mar 21, 2019
@maximede
Copy link

maximede commented Apr 17, 2019

I'm seeing a related issue with 2.6.0.

I have something like this :

resource "aws_autoscaling_group" "my_autoscaling_group" {
  name             = "my-asg"
  desired_capacity = "1"
  min_size         = "0"
  max_size         = "2"
    mixed_instances_policy {
    launch_template {
      launch_template_specification {
        launch_template_id = "${aws_launch_template.launch_template.id}"
        version            = "$Latest"
      }
    override {
        instance_type = "t2.small"
      }

      override {
        instance_type = "t2.large"
      }
  }
  instances_distribution {
      on_demand_base_capacity                  = "${var.on_demand_size}"
      on_demand_percentage_above_base_capacity = 0
    }
}

Originally, on_demand_size was set to 1. If I try to set it to 0, terraform doesn't detect any change in the plan and the apply is not changing anything either

@primeroz
Copy link

primeroz commented Oct 17, 2019

I am still having this issue with provider 2.19

whenever i set the on_demand_base_capacity to 0 ( or not set it at all ) it actually get set to 1 in the plan / apply phase

setting it to anything > 1 works as expected

--EDIT

apparently ( from #7938 (comment) ) my problem was that i was trying to change an already created ASG )

@scalp42
Copy link
Contributor

scalp42 commented Dec 10, 2019

I can confirm, we had to delete the ASG itself by hand to have Terraform correctly re-create it with the changes made to instances_distribution.

@tatodorov
Copy link

I can confirm this issue still exist with provider aws v2.62.0.
When I set "on_demand_base_capacity" to 1, it is updated successfully.
When I set it back to 0, Terraform does not detect the change.
As a workaround if I set "on_demand_base_capacity" from 1 to 2 and then from 2 to 0, the change is detected and applied successfully.

@anGie44 anGie44 self-assigned this Jun 3, 2020
@anGie44
Copy link
Contributor

anGie44 commented Jun 4, 2020

looking into this more, it's seems this line here in the autoscaling_group resource's DiffSuppressFunc for the instances_distribution attribute affects the on_demand_capacity update from 1 -> 0 (and not 2 -> 0) https://github.com/terraform-providers/terraform-provider-aws/blob/47a7974e2458920595eaa56ca29c43d66a03ae11/aws/resource_aws_autoscaling_group.go#L100

While this pattern has been used for other schema.TypeList attributes, such that the values passed to a DiffSuppressFunc reference the count, it seems in this case of a nested List, the values passed to the DiffSuppressFunc are the values associated with attributes in the instances_distribution list, in this case e.g. on_demand_capacity, and are interpreted literally; thus, a value of "0" will cause the diff suppression

with this being said, we could remove the DiffSuppressFunc but be forced to adjust the attributes to Computed for example, or maybe add a DiffSuppressFunc per attribute in the List to disable drift detection, or perhaps there's a known workaround for this use case. I'll circle back here with updated findings/proposed fix.

@anGie44
Copy link
Contributor

anGie44 commented Jun 8, 2020

The fix for this has been merged and will release with version 2.66.0 of the Terraform AWS Provider, expected in this week's release.

@ghost
Copy link

ghost commented Jun 12, 2020

This has been released in version 2.66.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

@ghost
Copy link

ghost commented Jul 8, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked and limited conversation to collaborators Jul 8, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.