provider/aws: aws_security_group_rule fails with protocol 50 or 51

[sarkis]

Terraform Version

0.6.16

Affected Resource(s)

• aws_security_group_rule

Terraform Configuration Files

resource "aws_security_group" "vpn" {
  description = "Security Group for East to West VPN"
  vpc_id = "${aws_vpc.us-east-1-doximity.id}"
}

resource "aws_security_group_rule" "vpn_protocol_50" {
  type = "ingress"
  from_port = 0
  to_port = 0
  protocol = "50"
  cidr_blocks = ["${aws_eip.us-west-2a-vpn.public_ip}/32"]
  security_group_id = "${aws_security_group.vpn.id}"
}

resource "aws_security_group_rule" "vpn_protocol_51" {
  type = "ingress"
  from_port = 0
  to_port = 0
  protocol = "51"
  cidr_blocks = ["${aws_eip.us-west-2a-vpn.public_ip}/32"]
  security_group_id = "${aws_security_group.vpn.id}"
}

Debug Output

https://gist.github.com/sarkis/2909062add54496b4dba0a69eaecd375

Panic Output

N/A

Expected Behavior

Security group rule created / updated in state

Actual Behavior

400 error from AWS

Steps to Reproduce

1. Set up an aws_security_group_rule with protocol set to any of: 50, "50", 51, "51", "ah", "esp"
2. terraform apply

Important Factoids

Nothing unique to our infra, we are running in EC2 VPC and have not had this issue until 0.6.16.

References

Are there any other GitHub issues (open or closed) or Pull Requests that should be linked here? For example:
Couldn't find any issues that directly relate to this one... Although, since we didn't have these problems with 0.6.14, it may be directly related to this PR:

#5881

[jrnt30]

I had lost track of this, but had a similar issue. I think there may be an issue with the AWS endpoint if memory serves me correctly where it was throwing an error even if I used it directly.

[sarkis]

@jrnt30 You are right about AWS endpoint. Here is what it looks like when I try to add "ah" or "esp" via aws cli:

$ aws ec2 authorize-security-group-ingress --group-id sg-9c09a2e7 --protocol ah --port 0 --cidr <some-ip>/32

protocol parameter should be one of: tcp|udp|icmp|all or any valid protocol number.

Notice: "or any valid protocol number", when I try the above using 51 (the ah protocol number vs "ah") it does work.

I'm positive now, the problem started from the PR which is converting 50 and 51 to "esp" and "ah" - #5881. At least for authorize-security-group-ingress (http://docs.aws.amazon.com/cli/latest/reference/ec2/authorize-security-group-ingress.html) aws endpoint is expecting "esp" or "ah" to be the actual protocol numbers.

[jrnt30]

That is the AWS CLI and I have seen similar things. I can try after work, but I am pretty sure if you try and do the same thing via the AWS GO SDK it was failing for me...but I am not 100% on that.

I believe I had tried removing that mapping (to ESP) for this explicit reason but got distracted from the issue.

[sarkis]

@jrnt30 thanks! I should take the opportunity to get aws-go-sdk installed and test this out - I assume the aws endpoint behavior is going to be the same... (I hope!) - I'll update with results if I can get to it before you :)

[sarkis]

Modifying the example from aws go sdk docs:

package main

import (
        "fmt"

        "github.com/aws/aws-sdk-go/aws"
        "github.com/aws/aws-sdk-go/aws/session"
        "github.com/aws/aws-sdk-go/service/ec2"
)

func main() {
        svc := ec2.New(session.New())

        params := &ec2.AuthorizeSecurityGroupIngressInput{
                CidrIp:     aws.String("0.0.0.0/0"),
                DryRun:     aws.Bool(false),
                FromPort:   aws.Int64(0),
                GroupId:    aws.String("sg-9673d9ed"),
                GroupName:  aws.String("test"),
                IpProtocol: aws.String("ah"),
                ToPort:     aws.Int64(0),
        }
        resp, err := svc.AuthorizeSecurityGroupIngress(params)

        if err != nil {
                // Print the error, cast err to awserr.Error to get the Code and
                // Message from an error.
                fmt.Println(err.Error())
                return
        }

        // Pretty-print the response data.
        fmt.Println(resp)
}

The above compiles and when run results in the following error:

$ ./test
InvalidParameterValue: Invalid value 'ah' for IP protocol. Unknown protocol.
        status code: 400, request id: d6a4b11c-53b2-4bcb-986f-66b79006dd66

Worth noting that even the go sdk docs say:
http://docs.aws.amazon.com/sdk-for-go/api/service/ec2.html#type-IpPermission
IpProtocol *string
The IP protocol name (for tcp, udp, and icmp) or number (see Protocol Numbers (http://www.iana.org/assignments/protocol-numbers/protocol-numbers.xhtml)).

[pecastro]

In our case we use a aws_security_group resource where we describe several ingress rules rather than a aws_security_group_rule and we see a different manifestation of the issue.
We can successfully terraform apply and all the ingress rules will be created in AWS but everytime we ran plan or apply after, terraform will redo all the SG's again. It apparently isn't recording the ingress bits related to the esp or ah protocols in its state file.
We do use the numeric id of the protocol in the rules and if we try to use the string version then we do see errors similar to the ones mentioned in comments above.

InvalidParameterValue: Invalid value 'esp' for IP protocol. Unknown protocol. status code: 400, request id: xxxxxxxxxx

In any case it's not a show stopper but it's noisy... :(

[mootpt]

This will be fixed in 0.7.5

[bgupta]

Note for posterity in case anyone lands here, because tf keeps reapplying your security group if it has an protocol ESP(50) rule. Typically "all ports" is -1 to -1. AWS has it as 0 to 0. (I suspect this is because ESP isn't "port-aware". If you just change your code to from port 0 to port 0, it will resolve your issues.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.