IAM Policy Attachment: "timeout while waiting for state to become '[success]'" #5447

jangrewe · 2016-03-03T15:43:36Z

I'm using v0.6.12 and when trying to apply some changes to an environment, i get these errors:

Error applying plan:

5 error(s) occurred:

* aws_iam_policy_attachment.auth_dynamodb: [WARN] Error attaching policy with IAM Policy Attachment auth_dynamodb:
– timeout while waiting for state to become '[success]'
* aws_iam_policy_attachment.uals_kinesis: [WARN] Error attaching policy with IAM Policy Attachment uals_kinesis:
– timeout while waiting for state to become '[success]'
* aws_iam_policy_attachment.cps_lambda: [WARN] Error attaching policy with IAM Policy Attachment cps_lambda:
– timeout while waiting for state to become '[success]'
* aws_iam_policy_attachment.cps_dynamodb: [WARN] Error attaching policy with IAM Policy Attachment cps_dynamodb:
– timeout while waiting for state to become '[success]'
* aws_iam_policy_attachment.reco_dynamodb: [WARN] Error attaching policy with IAM Policy Attachment reco_dynamodb:
– timeout while waiting for state to become '[success]'

The policies are attached to the role when i check through the web console, so there's so issue with TF figuring this out, as it seems...

The text was updated successfully, but these errors were encountered:

anosulchik · 2016-03-03T17:30:31Z

+1

elisehuard · 2016-03-04T11:08:32Z

+1

stack72 · 2016-03-04T11:38:26Z

Hi @jangrewe,

Can you post a sample configuration to see if i can reproduce this?

Paul

jangrewe · 2016-03-04T11:42:38Z

Hi @stack72,
unfortunately not right now, as i won't be in the office until monday.
But it's a simple inline policy that gets attached to a custom role, nothing fancy. It was working fine with older versions, as this new attachment was just a copy of a previous one - and then i tainted the whole role to see if it works when being recreated, which caused the error to happen for all attachments, event those that worked fine before (as they had to be recreated, too).

stack72 · 2016-03-04T11:48:06Z

ok, when you are back in the office, if you can, please post a snippet of your code. I have tried a few attachments (rather than inline) and they seem to work

I'd like to try and help diagnose this

Paul

faizhasim · 2016-03-05T16:25:36Z

Similar issue here. Works pre-v0.6.12, but not on v.0.6.12.

resource "aws_iam_role_policy" "aws_iam_role" {
  name = "aws_iam_role"
  role = "${aws_iam_role.ecs_instance_role.id}"
  policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "elasticloadbalancing:Describe*",
        "elasticloadbalancing:DeregisterInstancesFromLoadBalancer",
        "elasticloadbalancing:RegisterInstancesWithLoadBalancer",
        "ec2:Describe*",
        "ec2:AuthorizeSecurityGroupIngress"
      ],
      "Effect": "Allow",
      "Resource": "*"
    }
  ]
}
EOF
  lifecycle {
    create_before_destroy = true
  }
}

resource "aws_iam_role" "ecs_instance_role" {
  name = "ecs_instance_role"
  assume_role_policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Principal": {"AWS": "*"},
      "Effect": "Allow",
      "Sid": ""
    },
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "Service": "ec2.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    },
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "Service": "ecs.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
EOF
  lifecycle {
    create_before_destroy = true
  }
}

resource "aws_iam_policy_attachment" "ecs_service_role" {
  name = "ecs_service_role"
  roles = ["${aws_iam_role.ecs_instance_role.name}"]
  policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceRole"
  lifecycle {
    create_before_destroy = true
  }
}

resource "aws_iam_policy_attachment" "ecs_service_ec2_role" {
  name = "ecs_service_ec2_role"
  roles = ["${aws_iam_role.ecs_instance_role.name}"]
  policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceforEC2Role"
  lifecycle {
    create_before_destroy = true
  }
}

resource "aws_iam_instance_profile" "aws_iam_role" {
  name = "aws_iam_role"
  roles = ["${aws_iam_role.ecs_instance_role.name}"]
  lifecycle {
    create_before_destroy = true
  }
}

I got this:

2016/03/06 00:26:08 [DEBUG] root: eval: *terraform.EvalWriteState
2016/03/06 00:26:08 [DEBUG] root: eval: *terraform.EvalApplyProvisioners
2016/03/06 00:26:08 [DEBUG] root: eval: *terraform.EvalIf
2016/03/06 00:26:08 [DEBUG] root: eval: *terraform.EvalUndeposeState
2016/03/06 00:26:08 [DEBUG] root: eval: *terraform.EvalWriteDiff
2016/03/06 00:26:08 [DEBUG] root: eval: *terraform.EvalIf
2016/03/06 00:26:08 [DEBUG] root: eval: *terraform.EvalSequence
2016/03/06 00:26:08 [DEBUG] root: eval: *terraform.EvalWriteStateTainted
2016/03/06 00:26:08 [DEBUG] root: eval: *terraform.EvalIf
2016/03/06 00:26:08 [DEBUG] root: eval: *terraform.EvalApplyPost
2016/03/06 00:26:08 [ERROR] root: eval: *terraform.EvalApplyPost, err: 1 error(s) occurred:

* aws_iam_policy_attachment.ecs_service_ec2_role: [WARN] Error attaching policy with IAM Policy Attachment ecs_service_ec2_role:
– timeout while waiting for state to become '[success]'
2016/03/06 00:26:08 [ERROR] root: eval: *terraform.EvalSequence, err: 1 error(s) occurred:

* aws_iam_policy_attachment.ecs_service_ec2_role: [WARN] Error attaching policy with IAM Policy Attachment ecs_service_ec2_role:
– timeout while waiting for state to become '[success]'
2016/03/06 00:26:08 [ERROR] root: eval: *terraform.EvalOpFilter, err: 1 error(s) occurred:

* aws_iam_policy_attachment.ecs_service_ec2_role: [WARN] Error attaching policy with IAM Policy Attachment ecs_service_ec2_role:
– timeout while waiting for state to become '[success]'
2016/03/06 00:26:08 [ERROR] root: eval: *terraform.EvalSequence, err: 1 error(s) occurred:

* aws_iam_policy_attachment.ecs_service_ec2_role: [WARN] Error attaching policy with IAM Policy Attachment ecs_service_ec2_role:
– timeout while waiting for state to become '[success]'
2016/03/06 00:26:08 [TRACE] [walkApply] Exiting eval tree: aws_iam_policy_attachment.ecs_service_ec2_role
2016/03/06 00:26:08 [DEBUG] vertex provider.aws (close), got dep: aws_iam_policy_attachment.ecs_service_ec2_role
2016/03/06 00:26:08 [DEBUG] vertex root, got dep: provider.aws (close)
2016/03/06 00:26:08 [DEBUG] vertex root, got dep: module.security_groups.output.loadbalancer_id
2016/03/06 00:26:08 [DEBUG] vertex root, got dep: var.vpc_name
2016/03/06 00:26:08 [DEBUG] vertex root, got dep: module.security_groups.provider.aws
2016/03/06 00:26:08 [DEBUG] vertex root, got dep: module.security_groups.var.vpc_id
2016/03/06 00:26:08 [DEBUG] vertex root, got dep: output.aws_elb.general-purpose-elb.dns_name
2016/03/06 00:26:08 [DEBUG] vertex root, got dep: output.aws_rds_cluster.unicorn.port
2016/03/06 00:26:08 [DEBUG] vertex root, got dep: var.ssl_certs_aws_access_key_id

stack72 · 2016-03-05T18:02:06Z

Hi @jangrewe, I believe this has been fixed in #5460 but hopefully a sample of your code will help me test that :)

brikis98 · 2016-03-06T09:06:51Z

Is there any workaround for the timeout while waiting for state to become '[success]' error until #5460 is merged and released?

jangrewe · 2016-03-07T08:48:10Z

Hi @stack72, thanks for the update!

I've tested the fix from #5460 (downloaded and overwrote wait.go), but sadly the issue is still the same.

Here's how i'm doing the policy attachment:

resource "aws_iam_policy" "cps_lambda" {
    name = "cps_lambda"
    path = "/"
    description = "[cps] Allow access to Lambda"
    policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "lambda:invokeFunction"
      ],
      "Resource": "*"
    }
  ]
}
EOF
}

resource "aws_iam_policy_attachment" "cps_lambda" {
    name = "cps_lambda"
    roles = ["${aws_iam_role.ecs_instance.name}"]
    policy_arn = "${aws_iam_policy.cps_lambda.arn}"
}

jangrewe · 2016-03-07T11:31:14Z

I can confirm that it's working fine with v0.6.11, though!

s-nakka · 2016-03-07T23:33:22Z

I do have a similar issue. I am on version 0.6.12. terraform plan looks good, but apply return error.

resource "aws_lambda_function" "bobbyfox_test" {
function_name = "${var.function_name}"
role = "${var.role}"
s3_bucket = "${var.s3_bucket}"
s3_key = "${var.s3_key}"
s3_object_version = "$LATEST"
memory_size = "${var.memory}"
timeout = "${var.timeout}"
handler = "${var.handler}"
runtime = "${var.runtime}"
vpc_config = {
security_group_ids = ["${var.security_group_ids}"]
subnet_ids = ["${var.subnet_ids}"]
}
}

aws_lambda_function.bobbyfox_test: Error creating Lambda function: timeout while waiting for state to become '[success]'

mxs · 2016-03-09T02:30:30Z

I can also confirm this on v0.6.12

The timeout happens but the policy attachment actually works as verified on aws console.

resource "aws_iam_role" "ecsInstanceRole" {
    name = "ecsInstanceRole"
    assume_role_policy = <<EOF
{
  "Version": "2008-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "Service": "ec2.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
EOF
}

resource "aws_iam_policy_attachment" "AmazonEC2ContainerServiceforEC2Role-attach" {
    name = "AmazonEC2ContainerServiceforEC2Role-attachment"
    roles = ["${aws_iam_role.ecsInstanceRole.name}"]
    policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceforEC2Role"
}

jennyfountain · 2016-03-09T15:01:06Z

I am seeing the same error. Version 0.6.12

3 error(s) occurred:

aws_iam_policy_attachment.elasticsearch_policy_attach3: [WARN] Error attaching policy with IAM Policy Attachment elasticsearch_policy_attach3:
– timeout while waiting for state to become '[success]'
aws_iam_policy_attachment.elasticsearch_policy_attach2: [WARN] Error attaching policy with IAM Policy Attachment elasticsearch_policy_attach2:
– timeout while waiting for state to become '[success]'
aws_iam_policy_attachment.elasticsearch_policy_attach1: [WARN] Error attaching policy with IAM Policy Attachment elasticsearch_policy_attach1:
– timeout while waiting for state to become '[success]'

resource "aws_iam_policy_attachment" "elasticsearch_policy_attach1" {
name = "elasticsearch_policy_attach1"
roles = ["${aws_iam_role.ESRole.name}"]
policy_arn = "arn:aws:iam::aws:policy/AmazonEC2ReadOnlyAccess"
}

resource "aws_iam_policy_attachment" "elasticsearch_policy_attach2" {
name = "elasticsearch_policy_attach2"
roles = ["${aws_iam_role.ESRole.name}"]
policy_arn = "arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess"
}
resource "aws_iam_policy_attachment" "elasticsearch_policy_attach3" {
name = "elasticsearch_policy_attach3"
roles = ["${aws_iam_role.ESRole.name}"]
policy_arn = "arn:aws:iam::aws:policy/IAMReadOnlyAccess"
}

bmurphy1976 · 2016-03-09T16:18:46Z

To add another data point, I just checked out and built 5160578 and the problem persists for us:

$ terraform --version
Terraform v0.6.13-dev (5160578e186230aecb251bdcfb0219cac35c9230)

Here's a snippet of the output. It's subtly different, it no longer says timeout but the errors are on the same entities and it still behaves like a timeout.

* aws_iam_policy_attachment.logcluster01-boot-BootDefault: [WARN] Error attaching policy with IAM Policy Attachment logcluster01-boot-BootDefault:
– Policy ("arn:aws:iam::SNIP:policy/BootDefault") not yet found

jen20 · 2016-03-09T20:43:17Z

Hi all! The error message elision here is part of a wider group of issues which is being tracked by a meta-issue over at #5537. However, it looks like there is a wider issue as observed by @bmurphy1976 which needs addressing also, so I will leave this issue open so that we can circle back once #5537 is resolved.

catsby · 2016-03-15T20:00:31Z

Hey all – the issue here should be resolved in #5538 – please let us know if you're on the latest master and are still seeing this.

Thanks!

bmurphy1976 · 2016-03-15T21:47:23Z

Unless I'm doing something wrong, I'm still seeing this:

$ terraform --version
Terraform v0.6.13-dev (a1f77891611f47ad0c561d16a67007f4419a2358)

Output:

* aws_iam_policy_attachment.logcluster01-boot-BootDefault: [WARN] Error attaching policy with IAM Policy Attachment logcluster01-boot-BootDefault:
– timeout while waiting for state to become '[success]'

The culprit states look like this:

resource "aws_iam_policy_attachment" "logcluster01-boot-BootDefault" {
    name = "logcluster01-boot-BootDefault"
    policy_arn = "arn:aws:iam::SNIP:policy/BootDefault"
    roles = ["${aws_iam_role.logcluster01-boot.name}"]
}

Am I doing something wrong? Could old state data in the terraform.tfstate file be causing me grief?

bmurphy1976 · 2016-03-15T21:56:59Z

My bad, it is working for me now. I neglected to update the version of Terraform inside the docker container where we run our tests.

ghost · 2020-04-27T02:15:57Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

jen20 added bug provider/aws labels Mar 3, 2016

stack72 added the waiting-response An issue/pull request is waiting for a response from the community label Mar 4, 2016

jen20 removed the waiting-response An issue/pull request is waiting for a response from the community label Mar 9, 2016

catsby closed this as completed Mar 15, 2016

bmurphy1976 mentioned this issue Mar 16, 2016

problems with aws_iam_policy_attachment on subsequent applications (v0.6.13-dev) #5646

Closed

hashibot mentioned this issue Jun 13, 2017

problems with aws_iam_policy_attachment on subsequent applications (v0.6.13-dev) hashicorp/terraform-provider-aws#106

Closed

ghost locked and limited conversation to collaborators Apr 27, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

IAM Policy Attachment: "timeout while waiting for state to become '[success]'" #5447

IAM Policy Attachment: "timeout while waiting for state to become '[success]'" #5447

jangrewe commented Mar 3, 2016

anosulchik commented Mar 3, 2016

elisehuard commented Mar 4, 2016

stack72 commented Mar 4, 2016

jangrewe commented Mar 4, 2016

stack72 commented Mar 4, 2016

faizhasim commented Mar 5, 2016

stack72 commented Mar 5, 2016

brikis98 commented Mar 6, 2016

jangrewe commented Mar 7, 2016

jangrewe commented Mar 7, 2016

s-nakka commented Mar 7, 2016

mxs commented Mar 9, 2016

jennyfountain commented Mar 9, 2016

bmurphy1976 commented Mar 9, 2016

jen20 commented Mar 9, 2016

catsby commented Mar 15, 2016

bmurphy1976 commented Mar 15, 2016

bmurphy1976 commented Mar 15, 2016

ghost commented Apr 27, 2020

IAM Policy Attachment: "timeout while waiting for state to become '[success]'" #5447

IAM Policy Attachment: "timeout while waiting for state to become '[success]'" #5447

Comments

jangrewe commented Mar 3, 2016

anosulchik commented Mar 3, 2016

elisehuard commented Mar 4, 2016

stack72 commented Mar 4, 2016

jangrewe commented Mar 4, 2016

stack72 commented Mar 4, 2016

faizhasim commented Mar 5, 2016

stack72 commented Mar 5, 2016

brikis98 commented Mar 6, 2016

jangrewe commented Mar 7, 2016

jangrewe commented Mar 7, 2016

s-nakka commented Mar 7, 2016

mxs commented Mar 9, 2016

jennyfountain commented Mar 9, 2016

bmurphy1976 commented Mar 9, 2016

jen20 commented Mar 9, 2016

catsby commented Mar 15, 2016

bmurphy1976 commented Mar 15, 2016

bmurphy1976 commented Mar 15, 2016

ghost commented Apr 27, 2020