SQS Queue created without Policy cannot have policy attached

[tsholmes]

When attempting to plan a config that attaches a policy to an SQS queue, the plan fails if the SQS queue does not already have a policy attached.

https://github.com/terraform-providers/terraform-provider-aws/blob/8994a3474b02dd6f76cc5ecef2d1cfc6ba2e5def/aws/resource_aws_sqs_queue_policy.go#L79-L82

Terraform Version

0.10.7
aws provider 1.2.0 (but bug still exists in master)

Affected Resource(s)

aws_sqs_queue_policy

Terraform Configuration Files

There are a few other resources, but they aren't relevant.

resource "aws_sqs_queue_policy" "sqs_policy" {
  queue_url = "${aws_sqs_queue.q.id}"

  policy = <<POLICY
{
  "Version": "2012-10-17",
  "Id": "${aws_sqs_queue.q.id}/SQSDefaultPolicy",
  "Statement": [
    {
      "Sid": "SidNNNNNNNNNNN",
      "Effect": "Allow",
      "Principal": {
        "AWS": "${module.svc.task_role_arn}"
      },
      "Action": [
        "SQS:SendMessage",
        "SQS:PurgeQueue"
      ],
      "Resource": "${aws_sqs_queue.q.arn}"
    }
  ]
}
POLICY
}

Expected Behavior

The plan succeeds with changes showing the policy being added.

Actual Behavior

The plan fails with SQS Queue policy not found for sqs-policy-https://sqs.us-west-2.amazonaws.com/NNNNNNNNN/sqs_queue

Steps to Reproduce

1. Create an SQS queue without a Policy attached
2. Add policy to terraform config and plan

[bflad]

Aside: For future travelers, the error message in the original issue is a little misleading. We recently updated the resource ID via state migration to drop the sqs-policy- prefix in #2544 (released in 1.6.0), which now adjusts the error message to just the URL. The aws_sqs_queue_policy resource was previously and continues to correctly check the appropriate queue URL in the SDK calls.

@tsholmes Thanks for reporting this! It seems like there might be an eventual consistency issue with SQS queue attributes on creation then immediate read. The SetQueueAttributes API documentation does indeed confirm this is a possibility, since a SQS queue policy is really a Policy attribute on the queue:

When you change a queue's attributes, the change can take up to 60 seconds for most of the attributes to propagate throughout the Amazon SQS system.

Given this is dependent on AWS factors outside anyone's control, I am guessing we have not run into this too much with the existing acceptance testing. I was only able to reproduce this in 1 time out of a bunch myself.

We might be able to do some cursory retry checking in there, which I'll try to add now.

[bflad]

PR submitted: #2739

[bflad]

This is fixed in master and will release with v1.8.0 of the provider!

[tsholmes]

Awesome. Thanks @bflad!

[bflad]

This has been released in terraform-provider-aws version 1.8.0. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!