Get error when updating SG embedded ingress rules with description #2069

takeda-joao · 2017-10-26T17:52:21Z

Hi,

I am using terraform v0.10.8 and aws provider 1.1.0 and I am getting the following error when updating a security group ingress rules adding description field.
I am able to update the SG, but needed to remove all rules manually. Then terraform works and updates the SG. When I try to update the SG again, the error happens.
This error does not happen if the SG does not have any rules with description field.

terraform command line
$ terraform apply -target

=== output of terraform
Error: Error applying plan:

1 error(s) occurred:

aws_security_group.client-demo-sg-devops: 1 error(s) occurred:
aws_security_group.****: Error revoking security group ingress rules: InvalidPermission.NotFound: The specified rule does not exist in this security group.
status code: 400, request id: 2bfe24f7-a980-4bbc-a58d-54e5935f57a6

Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.

takeda-joao · 2017-10-26T18:07:26Z

Weird.
Made more changes, and there is a SG that is updated fine, with description included.
This second SG has same number of ingress rules of the one getting error and does not error when updated.
Making more tests.

takeda-joao · 2017-10-26T18:27:00Z

Looks like it is related to the lenght of the description field content.
If the content lenght of the description field of all ingress rules in the SG is the same, the error does not happen.

bflad · 2017-10-26T19:15:44Z

Potentially related? #1959

takeda-joao · 2017-10-26T19:20:26Z

No, it is not the lenght.

takeda-joao · 2017-10-26T19:25:30Z

Yes, #1959 may be related.
From the second run of terraform apply, without changes, and getting error, noticed it is comparing rules with the last rule of the security group.
Thanks,

jaymecd · 2017-11-01T11:36:43Z

@takeda-joao good catch

colin-lyman · 2017-11-07T08:36:42Z

Same issue experienced here.

colin-lyman · 2017-11-07T08:44:13Z

Just tested AWS Provider 1.2 and issue still persists.

SmaranNA · 2017-12-07T07:42:20Z

Any update on this issue? Im also facing this issue in my implementation.

jaymecd · 2017-12-07T08:31:24Z

it was fixed in #1959 and released as part of v1.3.1

hugohenley · 2017-12-07T11:04:00Z

Same issue here. Waiting for v1.3.1 to test.

reznet · 2017-12-07T23:01:03Z

I believe this issue still exists. I created new security groups with terraform and when i re-run apply, terraform complains with

* module.webui.aws_security_group.web: 1 error(s) occurred:

* aws_security_group.web: Error revoking security group ingress rules: InvalidPermission.NotFound: The specified rule does not exist in this security group.
	status code: 400, request id: 20c88505-d009-42ba-bc30-c0444cef2e94

setting the trace flag and looking at the aws call, terraform is passing the wrong description to aws:

Action=RevokeSecurityGroupIngress&GroupId=sg-b25012c7&IpPermissions.1.FromPort=3389&IpPermissions.1.IpProtocol=tcp&IpPermissions.1.IpRanges.1.CidrIp=172.16.0.0%2F16&IpPermissions.1.IpRanges.1.Description=RDP&IpPermissions.1.ToPort=3389&Version=2016-11-15

terraform is trying to remove the rule for CIDR 172.16.0.0/16 and description RDP, but the description for that CIDR block is actually RDP from Workspaces. There is a rule for a separate CIDR block whose description is RDP

Here's a snippet from my security group:

  ingress {
      description = "RDP"
      from_port = 3389
      to_port = 3389
      protocol = "tcp"
      cidr_blocks = ["10.0.0.0/8"]
  }

  ingress {
      description = "RDP from Workspaces"
      from_port = 3389
      to_port = 3389
      protocol = "tcp"
      cidr_blocks = ["${var.workspaces_cidr_blocks}"]
  }

From the terraform plan output, I see that terraform thinks it needs to delete the rule for RDP and create a rule for RDP For Workspaces, but it doesn't.

      ingress.1615219479.cidr_blocks.#:       "1" => "0"
      ingress.1615219479.cidr_blocks.0:       "172.16.0.0/16" => ""
      ingress.1615219479.description:         "RDP" => ""
      ingress.1615219479.from_port:           "3389" => "0"
      ingress.1615219479.ipv6_cidr_blocks.#:  "0" => "0"
      ingress.1615219479.protocol:            "tcp" => ""
      ingress.1615219479.security_groups.#:   "0" => "0"
      ingress.1615219479.self:                "false" => "false"
      ingress.1615219479.to_port:             "3389" => "0"
      ingress.2147368551.cidr_blocks.#:       "0" => "1"
      ingress.2147368551.cidr_blocks.0:       "" => "172.16.0.0/16"
      ingress.2147368551.description:         "" => "RDP from Workspaces"
      ingress.2147368551.from_port:           "" => "3389"
      ingress.2147368551.ipv6_cidr_blocks.#:  "0" => "0"
      ingress.2147368551.protocol:            "" => "tcp"
      ingress.2147368551.security_groups.#:   "0" => "0"
      ingress.2147368551.self:                "" => "false"
      ingress.2147368551.to_port:             "" => "3389"

Terraform version:

Terraform v0.11.1
+ provider.aws v1.5.0
+ provider.template v1.0.0
+ provider.terraform v1.0.2

takeda-joao · 2017-12-07T23:07:53Z

Problem persists.
I am also using provider.aws v1.5.0 and it makes impossible to run apply unless you remove de description field manually (and from terraform states).

tsiang · 2017-12-13T04:17:14Z

I'm seeing this problem as well. If I go ahead and delete the description in the console then delete the description field in tf state then terraform apply will run. But if I try to run apply again without changing anything, it will fail.

jaymecd · 2017-12-14T09:17:50Z

@reznet tnx for feedback, confirmed.
This issue exists only when using embedded ingress {} or egress {} blocks of aws_security_group resource.

Separate declaration of the group and rules works as expected and is more preferred way to do it.

resource "aws_security_group" "test" {
  name        = "test-rules"
  description = "test"
}

resource "aws_security_group_rule" "test_rule1" {
  type        = "ingress"
  description = "RDP"
  from_port   = "3389"
  to_port     = "3389"
  protocol    = "tcp"
  cidr_blocks = ["10.0.0.0/8"]

  security_group_id = "${aws_security_group.test.id}"
}

resource "aws_security_group_rule" "test_rule2" {
  type        = "ingress"
  description = "RDP from Workspaces"
  from_port   = "3389"
  to_port     = "3389"
  protocol    = "tcp"
  cidr_blocks = ["170.0.0.0/8"]

  security_group_id = "${aws_security_group.test.id}"
}

Could we mention embedded blocks in issue title? to make it more concrete?

@radeksimko wdyt?

karimodm · 2018-01-10T11:30:33Z

Same issue here.

MooreDerek · 2018-01-11T02:53:35Z

Seeing this too.

tf v.0.11.2
aws.provider 1.2.0 but it's fixed with aws-provider 1.6.0

jaymecd · 2018-01-11T13:24:52Z

Source of issue has been found - it's data struct returned by resourceAwsSecurityGroupIPPermGather()

Consider following TF resource:

resource "aws_security_group" "test" {
  name  = "test"

  ingress {
      description = "RDP 11111"
      from_port = 3389
      to_port = 3389
      protocol = "tcp"
      cidr_blocks = ["1.1.1.1/32"]
  }

  ingress {
      description = "RDP 22222"
      from_port = 3389
      to_port = 3389
      protocol = "tcp"
      cidr_blocks = ["2.2.2.2/32"]
  }
}

Local / Remote structs comparison (collapsed)

LOCAL:

[]interface {}{
    map[string]interface {}{
        "protocol":"tcp", 
        "ipv6_cidr_blocks":[]interface {}{}, 
        "self":false, 
        "description":"RDP 11111", 
        "from_port":3389, 
        "cidr_blocks":[]interface {}{"1.1.1.1/32"}, 
        "security_groups":*Set(map[string]interface {}(nil)), 
        "to_port":3389
    }, 
    map[string]interface {}{
        "self":false, 
        "description":"RDP 22222", 
        "to_port":3389, 
        "protocol":"tcp", 
        "security_groups":*Set(map[string]interface {}(nil)), 
        "from_port":3389, 
        "cidr_blocks":[]interface {}{"2.2.2.2/32"}, 
        "ipv6_cidr_blocks":[]interface {}{}
    }
}

REMOTE:

[]map[string]interface {}{
    map[string]interface {}{
        "description":"RDP 22222", 
        "from_port":3389, 
        "to_port":3389, 
        "protocol":"tcp", 
        "cidr_blocks":[]string{"1.1.1.1/32", "2.2.2.2/32"}
    }
}

So remote struct has only 1 item with detached description from cidr_blocks, that's why it could not be properly mapped to the local struct.

I'm going to work on the fix in the next days.

kenske · 2018-01-12T22:28:55Z

What's the best workaround for this in the meantime?

jaymecd · 2018-01-13T08:09:33Z

@kenske, use dedicated aws_security_group_rule resource

jaymecd · 2018-01-16T10:11:42Z

It's a little bit more complex that I thought before. @radeksimko @Ninir need ur advice.

Data structure of cidr_blocks, ipv6_cidr_blocks, prefix_list_ids & security_groups blocks from resource_aws_security_group.go should be upgraded from []string to []map[string]string to keep description together with value of the block or to slice of new internal type (I prefer this one):

type sgItem struct {
    Value string
    Description string
}

wdyt? Tnx

red97gst · 2018-01-31T00:32:43Z

any updates?

ScottSorrentino · 2018-02-02T20:41:43Z

@kenske I hit this same problem recently and worked around it by simply removing the descriptions from the similar port/protocol inline rules. After working around it, I found this open issue. I didn't want to go with the suggestion from @jaymecd since we use Terraform to ensure the SG rules exactly match what's in the definition. I don't think that's possible with aws_security_group_rule or, if it is, I haven't been able to find a way to have it clear out rules someone manually added via the AWS console/API.

I did need to enable debugging/tracing to figure out exactly what was being called (and failing) via RevokeSecurityGroupIngress, then I manually set the description on those rules in AWS so the next time terraform apply tried to remove them it would succeed. After that, and with no competing description fields, every terraform plan from that point forward was clean.

kenske · 2018-02-02T21:21:14Z

@ScottSorrentino It was kinda painful, but I did follow @jaymecd 's suggestion and it worked. It even worked for the scenario you are suggesting, clearing rules that were manually added.

sysadmin1139 · 2018-03-15T12:33:42Z

Since the business-case for inline rules seems to be missing, here it is in detail.

Inline-style declarations are a way of saying only these rules, where separate resource style says at least these rules. From a compliance point of view, only these rules is a policy enforcement mechanism, where at least these rules merely ensures approved functionality is present. This has very big impacts when it comes time for an audit.

When the auditors ask us:

Please describe how your firewall rules are enforced through change-control.

The use of inline style allows us to avoid having to build a complicated system of CloudTrail event ingestion and correlation with our change-request system to show that only approved changes are implemented and that change-drift is both detected and alarmed, if not outright prevented. Instead, we can give them the commit-log for our terraform repo. The auditors understand git-repos and merge-request approval flows. They have said, in essence:

Oh! Like Puppet, but for security-group definitions!

And that makes the whole process go so much easier. This is one of the biggest value-adds Terraform gives us in our infrastructure.

piersf · 2018-03-27T16:29:01Z

I just hit this issue today as well.
However, in my case where I have a rule like the below

resource "aws_security_group_rule" "DEFAULT_DO_NOT_USE-1" {
  type            = "ingress"
  from_port       = 0
  to_port         = 0
  protocol        = "-1"
  self            = "true"
  description = "Allow all incoming traffic from withing this security group"
  security_group_id = "${aws_security_group.DEFAULT_DO_NOT_USE.id}"
}

I can't use aws_security_group_rule as suggested above by @jaymecd as I will end up hitting a different bug explained in #1920

Any other solution?

piersf · 2018-03-28T12:15:15Z

Hi again,
After a lot of painful playing with this, I've concluded that the issue is when using the description field for either an ingress or egress rule and applying again after the first time. Below I write the steps that got me to this conclusion.

I have the below resource declaration (note description is commented out except in first ingress):

resource "aws_security_group" "bastion_priv_SG-master" {
  name        = "bastion_priv_SG-master"
  description = "Security group for bastion host"
  vpc_id = "${aws_vpc.us_n_virg_VPC-master-10_100_0_0-16.id}"
  
  # incoming
  ingress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["xx.xx.xx.xx/32"]
    description = "Allow all incoming traffic from xx.xx.xx.xx/32"
  }
  
  ingress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    self        = "true"
    #description = "Allow all incoming traffic from within this SG"
  }
  
  ingress {
    from_port   = 9100
    to_port     = 9100
    protocol    = "tcp"
    security_groups = ["${data.aws_security_group.app_priv_SG_QAMonitor.id}"]
    #description = "Allow all incoming traffic from app_priv_SG_QAMonitor SG"
  }

  # outgoing
  egress {
    from_port       = 0
    to_port         = 0
    protocol        = "-1"
    cidr_blocks     = ["0.0.0.0/0"]
    #description = "Allow all outgoing traffic from this SG to the Internet"
  }
  
  tags {
    Name = "bastion_priv_SG-master"
  }
}

Applying that the first time works fine and I see the expected changes in the security group. Strangely terraform deletes and recreates the first ingress rule. But it applies successfully. Below is the result of the apply action:

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  ~ aws_security_group.bastion_priv_SG-master
      ingress.1242592041.cidr_blocks.#:              "1" => "0"
      ingress.1242592041.cidr_blocks.0:              "xx.xx.xx.xx/32" => ""
      ingress.1242592041.description:                "" => ""
      ingress.1242592041.from_port:                  "0" => "0"
      ingress.1242592041.ipv6_cidr_blocks.#:         "0" => "0"
      ingress.1242592041.protocol:                   "-1" => ""
      ingress.1242592041.security_groups.#:          "0" => "0"
      ingress.1242592041.self:                       "false" => "false"
      ingress.1242592041.to_port:                    "0" => "0"
      ingress.1698735776.cidr_blocks.#:              "0" => "0"
      ingress.1698735776.description:                "" => ""
      ingress.1698735776.from_port:                  "9100" => "9100"
      ingress.1698735776.ipv6_cidr_blocks.#:         "0" => "0"
      ingress.1698735776.protocol:                   "tcp" => "tcp"
      ingress.1698735776.security_groups.#:          "1" => "1"
      ingress.1698735776.security_groups.3308286315: "sg-5477a429" => "sg-5477a429"
      ingress.1698735776.self:                       "false" => "false"
      ingress.1698735776.to_port:                    "9100" => "9100"
      ingress.502957524.cidr_blocks.#:               "0" => "1"
      ingress.502957524.cidr_blocks.0:               "" => "xx.xx.xx.xx/32"
      ingress.502957524.description:                 "" => "Allow all incoming traffic from xx.xx.xx.xx/32"
      ingress.502957524.from_port:                   "" => "0"
      ingress.502957524.ipv6_cidr_blocks.#:          "0" => "0"
      ingress.502957524.protocol:                    "" => "-1"
      ingress.502957524.security_groups.#:           "0" => "0"
      ingress.502957524.self:                        "" => "false"
      ingress.502957524.to_port:                     "" => "0"
      ingress.753360330.cidr_blocks.#:               "0" => "0"
      ingress.753360330.description:                 "" => ""
      ingress.753360330.from_port:                   "0" => "0"
      ingress.753360330.ipv6_cidr_blocks.#:          "0" => "0"
      ingress.753360330.protocol:                    "-1" => "-1"
      ingress.753360330.security_groups.#:           "0" => "0"
      ingress.753360330.self:                        "true" => "true"
      ingress.753360330.to_port:                     "0" => "0"


Plan: 0 to add, 1 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

aws_security_group.bastion_priv_SG-master: Modifying... (ID: sg-4183373a)
  ingress.1242592041.cidr_blocks.#:              "1" => "0"
  ingress.1242592041.cidr_blocks.0:              "xx.xx.xx.xx/32" => ""
  ingress.1242592041.description:                "" => ""
  ingress.1242592041.from_port:                  "0" => "0"
  ingress.1242592041.ipv6_cidr_blocks.#:         "0" => "0"
  ingress.1242592041.protocol:                   "-1" => ""
  ingress.1242592041.security_groups.#:          "0" => "0"
  ingress.1242592041.self:                       "false" => "false"
  ingress.1242592041.to_port:                    "0" => "0"
  ingress.1698735776.cidr_blocks.#:              "0" => "0"
  ingress.1698735776.description:                "" => ""
  ingress.1698735776.from_port:                  "9100" => "9100"
  ingress.1698735776.ipv6_cidr_blocks.#:         "0" => "0"
  ingress.1698735776.protocol:                   "tcp" => "tcp"
  ingress.1698735776.security_groups.#:          "1" => "1"
  ingress.1698735776.security_groups.3308286315: "sg-5477a429" => "sg-5477a429"
  ingress.1698735776.self:                       "false" => "false"
  ingress.1698735776.to_port:                    "9100" => "9100"
  ingress.502957524.cidr_blocks.#:               "0" => "1"
  ingress.502957524.cidr_blocks.0:               "" => "xx.xx.xx.xx/32"
  ingress.502957524.description:                 "" => "Allow all incoming traffic from xx.xx.xx.xx/32"
  ingress.502957524.from_port:                   "" => "0"
  ingress.502957524.ipv6_cidr_blocks.#:          "0" => "0"
  ingress.502957524.protocol:                    "" => "-1"
  ingress.502957524.security_groups.#:           "0" => "0"
  ingress.502957524.self:                        "" => "false"
  ingress.502957524.to_port:                     "" => "0"
  ingress.753360330.cidr_blocks.#:               "0" => "0"
  ingress.753360330.description:                 "" => ""
  ingress.753360330.from_port:                   "0" => "0"
  ingress.753360330.ipv6_cidr_blocks.#:          "0" => "0"
  ingress.753360330.protocol:                    "-1" => "-1"
  ingress.753360330.security_groups.#:           "0" => "0"
  ingress.753360330.self:                        "true" => "true"
  ingress.753360330.to_port:                     "0" => "0"
aws_security_group.bastion_priv_SG-master: Modifications complete after 2s (ID: sg-4183373a)

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

Now, if I apply again without changing anything in the resources, it will fail:

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  ~ aws_security_group.bastion_priv_SG-master
      ingress.1698735776.cidr_blocks.#:              "0" => "0"
      ingress.1698735776.description:                "" => ""
      ingress.1698735776.from_port:                  "9100" => "9100"
      ingress.1698735776.ipv6_cidr_blocks.#:         "0" => "0"
      ingress.1698735776.protocol:                   "tcp" => "tcp"
      ingress.1698735776.security_groups.#:          "1" => "1"
      ingress.1698735776.security_groups.3308286315: "sg-5477a429" => "sg-5477a429"
      ingress.1698735776.self:                       "false" => "false"
      ingress.1698735776.to_port:                    "9100" => "9100"
      ingress.2396394866.cidr_blocks.#:              "0" => "0"
      ingress.2396394866.description:                "Allow all incoming traffic from xx.xx.xx.xx/32" => ""
      ingress.2396394866.from_port:                  "0" => "0"
      ingress.2396394866.ipv6_cidr_blocks.#:         "0" => "0"
      ingress.2396394866.protocol:                   "-1" => ""
      ingress.2396394866.security_groups.#:          "0" => "0"
      ingress.2396394866.self:                       "true" => "false"
      ingress.2396394866.to_port:                    "0" => "0"
      ingress.502957524.cidr_blocks.#:               "1" => "1"
      ingress.502957524.cidr_blocks.0:               "xx.xx.xx.xx/32" => "xx.xx.xx.xx/32"
      ingress.502957524.description:                 "Allow all incoming traffic from xx.xx.xx.xx/32" => "Allow all incoming traffic from xx.xx.xx.xx/32"
      ingress.502957524.from_port:                   "0" => "0"
      ingress.502957524.ipv6_cidr_blocks.#:          "0" => "0"
      ingress.502957524.protocol:                    "-1" => "-1"
      ingress.502957524.security_groups.#:           "0" => "0"
      ingress.502957524.self:                        "false" => "false"
      ingress.502957524.to_port:                     "0" => "0"
      ingress.753360330.cidr_blocks.#:               "0" => "0"
      ingress.753360330.description:                 "" => ""
      ingress.753360330.from_port:                   "" => "0"
      ingress.753360330.ipv6_cidr_blocks.#:          "0" => "0"
      ingress.753360330.protocol:                    "" => "-1"
      ingress.753360330.security_groups.#:           "0" => "0"
      ingress.753360330.self:                        "" => "true"
      ingress.753360330.to_port:                     "" => "0"

Plan: 0 to add, 1 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

aws_security_group.bastion_priv_SG-master: Modifying... (ID: sg-4183373a)
  ingress.1698735776.cidr_blocks.#:              "0" => "0"
  ingress.1698735776.description:                "" => ""
  ingress.1698735776.from_port:                  "9100" => "9100"
  ingress.1698735776.ipv6_cidr_blocks.#:         "0" => "0"
  ingress.1698735776.protocol:                   "tcp" => "tcp"
  ingress.1698735776.security_groups.#:          "1" => "1"
  ingress.1698735776.security_groups.3308286315: "sg-5477a429" => "sg-5477a429"
  ingress.1698735776.self:                       "false" => "false"
  ingress.1698735776.to_port:                    "9100" => "9100"
  ingress.2396394866.cidr_blocks.#:              "0" => "0"
  ingress.2396394866.description:                "Allow all incoming traffic from xx.xx.xx.xx/32" => ""
  ingress.2396394866.from_port:                  "0" => "0"
  ingress.2396394866.ipv6_cidr_blocks.#:         "0" => "0"
  ingress.2396394866.protocol:                   "-1" => ""
  ingress.2396394866.security_groups.#:          "0" => "0"
  ingress.2396394866.self:                       "true" => "false"
  ingress.2396394866.to_port:                    "0" => "0"
  ingress.502957524.cidr_blocks.#:               "1" => "1"
  ingress.502957524.cidr_blocks.0:               "xx.xx.xx.xx/32" => "xx.xx.xx.xx/32"
  ingress.502957524.description:                 "Allow all incoming traffic from xx.xx.xx.xx/32" => "Allow all incoming traffic from xx.xx.xx.xx/32"
  ingress.502957524.from_port:                   "0" => "0"
  ingress.502957524.ipv6_cidr_blocks.#:          "0" => "0"
  ingress.502957524.protocol:                    "-1" => "-1"
  ingress.502957524.security_groups.#:           "0" => "0"
  ingress.502957524.self:                        "false" => "false"
  ingress.502957524.to_port:                     "0" => "0"
  ingress.753360330.cidr_blocks.#:               "0" => "0"
  ingress.753360330.description:                 "" => ""
  ingress.753360330.from_port:                   "" => "0"
  ingress.753360330.ipv6_cidr_blocks.#:          "0" => "0"
  ingress.753360330.protocol:                    "" => "-1"
  ingress.753360330.security_groups.#:           "0" => "0"
  ingress.753360330.self:                        "" => "true"
  ingress.753360330.to_port:                     "" => "0"

Error: Error applying plan:

1 error(s) occurred:

* aws_security_group.bastion_priv_SG-master: 1 error(s) occurred:

* aws_security_group.bastion_priv_SG-master: Error revoking security group ingress rules: InvalidPermission.NotFound: The specified rule does not exist in th
        status code: 400, request id: 98d8fa95-0aa2-40b8-851d-0a10bb25572b

Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.

Where is the issue..?
In the result of the second apply action above, note the ingress rule with number 2396394866
Terraform, somehow and from somewhere sees an ingress rule that tries to delete and which doesn't exist anywhere in my state file or my security group in AWS. And that's why it complains that the security group ingress rule does not exist.
Why and how is Terraform seeing something that doesn't exist in the .tf or the state file is the question.

And like that isn't enough, I can't get out of this situation even if I comment out the description from the ingress rule. It will still fail with the same error. And I suspect that's because the state file now is in an inconsistent state since it doesn't rollback.

I have to manually delete the security group resource from the state file, do a refresh, import it back and then create one by one the ingress and egress rules without the description.

Can't really say where the bug is here, but I'd suggest to anyone else not providing a description for nested ingress and egress rules if you don't want to get in this situation.

I am running Terraform v0.11.5

Thanks,

shanemeyers · 2018-03-30T06:03:12Z

Got bit by this tonight. Even if I commented out the description it still triggered the bug. Only after completely deleting the line did things start to work as expected.

Terraform v0.11.5
+ provider.aws v1.13.0

rajbettaswamy · 2018-04-04T17:47:30Z

Bug reintroduced again in

Terraform v0.11.5
Provider "aws" (1.13.0)

ibrahima · 2018-04-11T23:31:39Z

Was the bug actually ever fixed? I'm still seeing it too, although I can't figure out how to update my AWS provider (I'm on 1.6 and running terraform init does nothing to change that).

piersf · 2018-04-12T01:00:07Z

@ibrahima still haven't seen any updates about the bug.
As for the provide version, try deleting the file similar to .terraform/plugins/linux_amd64/terraform-provider-aws_v1.14.1_x4 which is what I have and then run init again. This will update the version of the provider.

ibrahima · 2018-04-12T01:08:45Z

Ah okay, I actually figured that out after posting the comment (kinda counterintuitive, because the terraform init message says that it installed the latest version, but oh well). And now I actually am experiencing more bugs rather than fewer :(

junkangli · 2018-04-12T01:09:08Z

To update the provider, you run terraform init -upgrade.

russellsherman · 2018-04-12T20:10:27Z

Experiencing this as well.

Terraform v0.11.5
+ provider.archive v1.0.3
+ provider.aws v1.11.0

pmacdougall · 2018-04-24T15:38:09Z

Just wanted to note that this also happens for the aws_default_security_group resource, where using individual aws_security_group_rule resources is not an option

reza · 2018-04-26T08:17:31Z

I face a similar issue when updating security group rules, it has nothing to do with description, it seems that the tf state file becomes out of sync with with the security group rules at apply time, it first removes/modifies the security rule then tries to add new properties.

$ terraform -version
Terraform v0.11.7

provider.aws v1.16.0
provider.null v1.0.0
provider.random v1.2.0
provider.template v1.0.0

Caspain · 2018-05-08T16:22:46Z

This is still present, i had manually delete all descriptions in aws and rerun the terraform script.

carlosescura · 2018-05-09T15:06:04Z

@Caspain I also had the same issue today. Deleting descriptions and applying again worked fine

bflad · 2018-05-10T20:12:37Z

Hi folks 👋 Sorry this has been a longstanding issue with the AWS provider. The fix for this should be contained in #4416 which will be released with v1.19.0 of the AWS provider, likely middle of next week.

Shout outs to @loivis (and @svanharmelen who submitted an earlier, likely correct PR, which I admittedly should have reviewed and merged sooner: #3628)

Given there were so many various issues surrounding this bug, I will be locking this issue (amongst all the others) to encourage any lingering issues/discussion to be fully described in new issue(s) for consolidation. Thanks for your understanding.

bflad mentioned this issue Oct 26, 2017

r/security_group_rule: parse description correctly #1959

Merged

paddycarver added the bug Addresses a defect in current functionality. label Nov 21, 2017

takeda-joao changed the title ~~Get error when updating SG ingress rules with description~~ Get error when updating SG embedded ingress rules with description Dec 18, 2017

radeksimko added the service/ec2 Issues and PRs that pertain to the ec2 service. label Jan 17, 2018

MooreDerek mentioned this issue Jan 30, 2018

Bug in aws_security_group with descriptions #3202

Closed

svanharmelen mentioned this issue Mar 5, 2018

r/security_group: improve PR #1587 to fix issue #2069 #3628

Closed

l3xicon mentioned this issue Mar 22, 2018

SG contents overwritten on each run when "self" rules are referenced #3872

Closed

pmacdougall mentioned this issue Apr 24, 2018

aws_default_security_group fails on a repetitive run when description is defined #3482

Closed

loivis mentioned this issue May 1, 2018

resource/security_group: fix rule gathering with description included #4416

Merged

bflad added this to the v1.19.0 milestone May 10, 2018

bflad closed this as completed in #4416 May 10, 2018

hashicorp locked as resolved and limited conversation to collaborators May 10, 2018

Get error when updating SG embedded ingress rules with description #2069

Get error when updating SG embedded ingress rules with description #2069

Comments

takeda-joao commented Oct 26, 2017

takeda-joao commented Oct 26, 2017

takeda-joao commented Oct 26, 2017

bflad commented Oct 26, 2017

takeda-joao commented Oct 26, 2017

takeda-joao commented Oct 26, 2017

jaymecd commented Nov 1, 2017

colin-lyman commented Nov 7, 2017

colin-lyman commented Nov 7, 2017

SmaranNA commented Dec 7, 2017

jaymecd commented Dec 7, 2017

hugohenley commented Dec 7, 2017

reznet commented Dec 7, 2017 • edited Loading

takeda-joao commented Dec 7, 2017

tsiang commented Dec 13, 2017

jaymecd commented Dec 14, 2017

karimodm commented Jan 10, 2018

MooreDerek commented Jan 11, 2018 • edited Loading

jaymecd commented Jan 11, 2018

kenske commented Jan 12, 2018

jaymecd commented Jan 13, 2018

jaymecd commented Jan 16, 2018

red97gst commented Jan 31, 2018

ScottSorrentino commented Feb 2, 2018

kenske commented Feb 2, 2018

sysadmin1139 commented Mar 15, 2018

piersf commented Mar 27, 2018

piersf commented Mar 28, 2018

shanemeyers commented Mar 30, 2018 • edited Loading

rajbettaswamy commented Apr 4, 2018

ibrahima commented Apr 11, 2018 • edited Loading

piersf commented Apr 12, 2018

ibrahima commented Apr 12, 2018

junkangli commented Apr 12, 2018

russellsherman commented Apr 12, 2018

pmacdougall commented Apr 24, 2018

reza commented Apr 26, 2018 • edited Loading

Caspain commented May 8, 2018

carlosescura commented May 9, 2018

bflad commented May 10, 2018

reznet commented Dec 7, 2017 •

edited

Loading

MooreDerek commented Jan 11, 2018 •

edited

Loading

shanemeyers commented Mar 30, 2018 •

edited

Loading

ibrahima commented Apr 11, 2018 •

edited

Loading

reza commented Apr 26, 2018 •

edited

Loading