Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cloudformation stack update timeout doesn't work #4432

Closed
raylu opened this issue May 2, 2018 · 2 comments
Closed

cloudformation stack update timeout doesn't work #4432

raylu opened this issue May 2, 2018 · 2 comments
Labels
bug Addresses a defect in current functionality. service/cloudformation Issues and PRs that pertain to the cloudformation service. stale Old or inactive issues managed by automation, if no further action taken these will get closed.

Comments

@raylu
Copy link

raylu commented May 2, 2018

Terraform Version

$ terraform -v
Terraform v0.11.7
+ provider.aws v1.12.0

Affected Resource(s)

  • aws_cloudformation_stack

Terraform Configuration Files

resource "aws_cloudformation_stack" "asg" {
  name = "${var.deploy}-web-asg"

  template_body = <<TEMPLATE
Resources:
  autoScalingGroup:
    Type: AWS::AutoScaling::AutoScalingGroup
    Properties:
      MinSize: ${var.instance_count}
      MaxSize: 20
      Cooldown: 180
      LaunchConfigurationName: '${module.launch-config.name}'
      MetricsCollection:
      - Granularity: 1Minute
      TargetGroupARNs:
      - ${var.target_group}
      VPCZoneIdentifier: [${join(",", var.subnet_ids)}]
    UpdatePolicy:
      AutoScalingRollingUpdate:
        MaxBatchSize: ${var.instance_count}
        MinInstancesInService: ${var.instance_count}
        PauseTime: PT0S
        SuspendProcesses:
        - HealthCheck
        - ReplaceUnhealthy
        - AZRebalance
        - AlarmNotification
        - ScheduledActions
Outputs:
  ASGName:
    Description: Name of the auto scaling group
    Value: !Ref 'autoScalingGroup'
TEMPLATE

  tags = "${merge(map("Role", "web"), var.tags)}"
}

Debug Output

sorry, I don't have a way of reproducing the exact failure because normally this cloudformation stack updates quickly but in this case, a lifecycle event hung the update indefinitely

Panic Output

Expected Behavior

the terraform apply bails with an error after 30 minutes

Actual Behavior

terraform update kept running until I interrupted it
module.zym.celery.aws_cloudformation_stack.asg-b: Still modifying... (ID: arn:aws:cloudformation:us-east-1:371589...b/82a95fd0-e769-11e7-b5aa-50d5ca6e60ae, 59m40s elapsed)

Steps to Reproduce

  1. set up a failing instance launch lifecycle hook on an ASG
  2. terraform apply a cloudformation stack change that replaces the instances in the ASG

Important Factoids

we have no override on the timeouts, so I expected the default of 30 minutes

the state file shows

                        "meta": {
                            "e2bfb730-ecaa-11e6-8f88-34363bc7c4c0": {
                                "create": 1800000000000,
                                "delete": 1800000000000,
                                "update": 1800000000000
                            }
                        },

which is 30 minutes in nanoseconds

References

@bflad bflad added bug Addresses a defect in current functionality. service/cloudformation Issues and PRs that pertain to the cloudformation service. labels May 12, 2018
@github-actions
Copy link

github-actions bot commented May 1, 2020

Marking this issue as stale due to inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 30 days it will automatically be closed. Maintainers can also remove the stale label.

If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!

@github-actions github-actions bot added the stale Old or inactive issues managed by automation, if no further action taken these will get closed. label May 1, 2020
@ghost
Copy link

ghost commented Jul 1, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked and limited conversation to collaborators Jul 1, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Addresses a defect in current functionality. service/cloudformation Issues and PRs that pertain to the cloudformation service. stale Old or inactive issues managed by automation, if no further action taken these will get closed.
Projects
None yet
Development

No branches or pull requests

2 participants