Terraform crashed on apply

[pbitty]

I was trying to apply changes to AWS resources (primarily security groups) and Terraform crashed. Here's a gist of crash.log.

I hope this helps. If there's anything else I can answer, let me know.

[stack72]

@pbitty - please can you post your terraform config (without secrets) so we can try and look at this. Also, were you trying to change infra or was this an initial creation?

[phinze]

Relevant log lines:

2016/02/24 16:29:16 [DEBUG] terraform-provider-aws: github.com/hashicorp/terraform/builtin/providers/aws.resourceAwsSecurityGroupRuleRead(0xc820414540, 0x96c760, 0xc8203ca000, 0x0, 0x0)
2016/02/24 16:29:16 [DEBUG] terraform-provider-aws:     /private/tmp/terraform20160205-18753-13bx7e7/terraform-0.6.11/src/github.com/hashicorp/terraform/builtin/providers/aws/resource_aws_security_group_rule.go:218 +0x17c9
2016/02/24 16:29:16 [DEBUG] terraform-provider-aws: github.com/hashicorp/terraform/builtin/providers/aws.resourceAwsSecurityGroupRuleCreate(0xc820414540, 0x96c760, 0xc8203ca000, 0x0, 0x0)
2016/02/24 16:29:16 [DEBUG] terraform-provider-aws:     /private/tmp/terraform20160205-18753-13bx7e7/terraform-0.6.11/src/github.com/hashicorp/terraform/builtin/providers/aws/resource_aws_security_group_rule.go:154 +0xaab
2016/02/24 16:29:16 [DEBUG] terraform-provider-aws: github.com/hashicorp/terraform/helper/schema.(*Resource).Apply(0xc8204bc940, 0xc82038b830, 0xc8202cf5f0, 0x96c760, 0xc8203ca000, 0x10101, 0x0, 0x0)
2016/02/24 16:29:16 [DEBUG] terraform-provider-aws:     /private/tmp/terraform20160205-18753-13bx7e7/terraform-0.6.11/src/github.com/hashicorp/terraform/helper/schema/resource.go:145 +0x28e

Indicates crash was here: https://github.com/hashicorp/terraform/blob/master/builtin/providers/aws/resource_aws_security_group_rule.go#L218

Looks like a pre-0.6.12 build is running but I'm not sure the code has changed much since then.

[pbitty]

I'm running version Terraform v0.6.11.

The crash happens on initial creation. If I re-run, I get these errors:

Error applying plan:

11 error(s) occurred:

* aws_security_group.k8s_etcd: Error authorizing security group ingress rules: InvalidGroup.NotFound: Unable to find group 'sg-22312748'
    status code: 400, request id:
* aws_security_group_rule.k8s_servers_worker_ingress_udp: unexpected EOF
* aws_security_group_rule.k8s_workers_worker_ingress_tcp: unexpected EOF
* aws_security_group_rule.k8s_workers_worker_ingress_icmp: unexpected EOF
* aws_security_group_rule.k8s_servers_worker_ingress_icmp: unexpected EOF
* aws_security_group_rule.k8s_workers_server_ingress_udp: unexpected EOF
* aws_security_group_rule.k8s_workers_server_ingress_icmp: unexpected EOF
* aws_security_group_rule.k8s_servers_worker_ingress_tcp: unexpected EOF
* aws_security_group_rule.k8s_workers_worker_ingress_udp: unexpected EOF
* aws_route53_record.k8s_server: unexpected EOF
* aws_security_group_rule.k8s_workers_server_ingress_tcp: unexpected EOF

[pbitty]

The config is somewhat large. I'm trying to reduce it to just the parts that make it crash. Would that help or do you prefer to see the whole thing?

[pbitty]

I've boiled it down to this config:

resource "aws_security_group" "alpha" {
  name = "terraform-test-alpha"
  description = "Terraform Test"
}

resource "aws_security_group_rule" "alpha_self" {
  type = "ingress"
  from_port = 0
  to_port = 65535
  protocol = "tcp"
  security_group_id = "${aws_security_group.alpha.id}"
  self = true
}

Here's another gist with crash.log and some command output.

[pbitty]

I'm getting a different error (not a crash) when trying to allow traffic from one SG to another:

* aws_security_group_rule.alpha_ingress_from_beta: Error authorizing security group rule type ingress: InvalidGroup.NotFound: Unable to find group 'sg-a63f29cc'
    status code: 400, request id:

In this case, I'm not using the self attribute, instead referring to another SG. It might be related to the same issue. Here's a gist.

[pbitty]

This seems to only happen in EC2-Classic. If I add a vpc_id to the security groups in all the examples, it works as expected.

[pbitty]

I just tried the same 'self-referencing SG' config with 0.6.12 and the same thing happens.

[jen20]

Thanks for the detailed repro @pbitty. I can see the issue now on one of our accounts with EC2 classic enabled, and will get it fixed!

[jen20]

Hi @pbitty - I've opened up #5329 which should address this issue, and has a regression test using your reproduction of the problem. If you are able to test development builds it would be great to get a 👍/👎 on whether this fixes the issue for you?

[pbitty]

Sure! I can test tomorrow morning. Thanks for looking at it so quickly.

[pbitty]

@jen20 I just tested this and it works as expected. Thanks again.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.