AWS: attach new security group to eth0 in an existing instance with 2 interfaces.

[jllucas]

Hi all,

In my AWS infrastructure I've created a new security group (newsecgroup).
When attaching it to an already running instance with 2 interfaces, terraform raises the following error:

* aws_instance.myinstance: InvalidInstanceID: There are multiple interfaces attached to instance 'i-XXXXX'. Please specify an interface ID for the operation instead.
    status code: 400, request id: []

I only want to attach it to eth0, and not to eth1.

Here is my terraform definition:

resource "aws_instance" "myinstance" {
  ...
  vpc_security_group_ids = [ "${aws_security_group.default.id}", "${aws_security_group.newsecgroup.id}" ]
 ...

resource "aws_network_interface" "myinstance-service" {
    ...
    security_groups = [ "${aws_security_group.default.id}" ]
   ...
}

If I don't define a resource for eth0 (default network interface), how can I specify the interface ID?
Any clue?

Thanks!

EDIT --> If I attach the security group using AWS console, and execute terraform plan, terraform shows "No changes".

[zzzuzik]

+1

[Kuberchaun]

I'm seeing something similar that happens when trying to set monitoring = true on an ec2 instance resource. I have two network interfaces eth0 and eth1. Also in my case the ec2 was brought up with monitoring set to false. A later change set it to true in the tf file which is when this error pops up.

Terraform v0.6.16

InvalidInstanceID: There are multiple interfaces attached to instance 'i-XXXXX'. Please specify an interface ID for the operation instead.
status code: 400, request id: XXXXXXXXXXX

[Ehekatl]

same problem here
I need fixed private ip and create before destroy on instance
so I create a secondary network interface with fixed private ip, but when I apply changes to aws_instance, the error happen

[kntait]

Is there any update on this issue? We are experiencing the same issue when trying to add additional security groups to the default network interface built by the "aws_instance" resource.

Terraform version : 0.8.4

Error:
InvalidInstanceID: There are multiple interfaces attached to instance 'xxxx'. Please specify an interface ID for the operation instead.

For network interfaces built with the aws_network_interface resource we are able to add and remove security groups.

[jinnko]

Have hit this issue and believe I understand the issue, however haven't had a chance to dig into the code to fix it.

Terraform assumes there's a single interface on systems, and therefore appears to makes API calls to the effect of aws ec2 modify-instance-attributes, which works when there's a single interface. However when there's multiple interfaces, for example applied using code such as the following, this fails.

# Additional interface for an-ec2-instance-1 interface
resource "aws_network_interface" "additional-network-interface-on-an-ec2-instance-1" {
...
    security_groups = [
        "${aws_security_group.staging-eth1-common.id}",
        "${aws_security_group.staging-eth1-custom.id}"
    ]
    attachment {
        instance = "${aws_instance.an-ec2-instance-1.id}"
        device_index = 1
    }
}

# an-ec2-instance-1
resource "aws_instance" "an-ec2-instance-1" {
...
  vpc_security_group_ids = [
    "${aws_security_group.staging-eth0-common.id}",
    "${aws_security_group.staging-eth0-custom.id}"
  ]
...
}

I believe what terraform must do is determine the network interface id for the resource that is receiving the change, then apply the security groups on the network interface using something similar to aws ec2 modify-network-interface-attribute to change the security groups. For example if you're modifying the security groups on the main instance, the main network id would be used, however if the change is on the first network interface resource in the example above, then you'd apply to that interface.

The following is a quick and dirty script that provides a way around the issue for now, but only on the primary interface of the instance. It would need to be extended if the change is applied to the secondary interface.

#!/bin/sh -e

set -o pipefail

if [ -z $1 ]; then
  cat <<EOM
Usage: $(basename "$0") aws_security_group.THE_SG_NAME

The script assumes that the security group has been applied to the terraform
aws_instance resources and will therefore:

1) crudely extract the instance names for all instances with the SG defined
   in the terraform code.
2) terraform query the instance for its network interface id
3) terraform query the security group for the id
4) apply the existing and new security group to the primary network interface
   of the instances

Example to apply a new "staging-eth0-common" SG to instances:

    ./$(basename "$0") aws_security_group.staging-eth0-common

EOM
  exit 1
fi
SG="$1"

SG_ID=$(terraform state show $SG | awk '/^id[ | ]/ {print $3 }')
HOSTS_WITH_SG=$(git grep -B 30 "${SG}.id" | awk '/resource "aws_instance"/ { gsub(/"/, ""); print $3 }')

for HOST_WITH_SG in $HOSTS_WITH_SG; do
  unset VPC_SECURITY_GROUP_IDS
  eval $(terraform state show aws_instance.${HOST_WITH_SG} | awk '/^network_interface_id/ { print "export NETWORK_INTERFACE_ID=" $3 ";\n" }; /^vpc_security_group_ids.[0-9]/ { print "export VPC_SECURITY_GROUP_IDS=\"\$VPC_SECURITY_GROUP_IDS " $3 "\"" }')

  VPC_SECURITY_GROUP_IDS="$VPC_SECURITY_GROUP_IDS $SG_ID"

  aws ec2 modify-network-interface-attribute \
    --network-interface-id ${NETWORK_INTERFACE_ID} \
    --groups $VPC_SECURITY_GROUP_IDS
done

[grubernaut]

Hi all,

I came across this issue when solving #12933. The error:

 aws_instance.myinstance: InvalidInstanceID: There are multiple interfaces attached to instance 'i-XXXXX'. Please specify an interface ID for the operation instead.
    status code: 400, request id: []

Actually comes from Terraform attempting to set the source_dest_check attribute on the instance when there are multiple network interfaces on the instance, and the interface at device_index 0 isn't the default interface.
Unfortunately it's a fairly cryptic error message 😢

That being said, after #12933 lands, users should be able to create their network interfaces directly in the instance resource, thus eliminating the need for the vpc_security_group_id's attribute inside the instance. Allowing full control over the security groups attached to each network interface specifically.

Please let me know if there are any other concerns with this issue, and I can try and address them in a future patch. ❤️

[cnoffsin]

@grubernaut Can you expand on this please?:

That being said, after #12933 lands, users should be able to create their network interfaces directly in the instance resource, thus eliminating the need for the vpc_security_group_id's attribute inside the instance. Allowing full control over the security groups attached to each network interface specifically.
Please let me know if there are any other concerns with this issue, and I can try and address them in a future patch. ❤️

We have this issue, multiple interfaces and need to make changes to the the interface security groups without having to rebuild the instance every time.

Can you give a code snippet that would work? It's hard to tell if 12933 got merged into 0.9.2 or not.

[grubernaut]

Hi @cnoffsin,

#12933 has not been merged into TF 0.9.2 and has not been released yet. There are still some quirks that we need to solve before it can be released, and unfortunately other more pressing items have taken a higher priority.

Once the patch in #12933 is merged, full documentation of the optional network_interface configuration block will be added, along with code examples.

[cnoffsin]

@grubernaut
Thank you for your quick response.

This is troubling because we are not sure what work-around we can even make when making security group changes to these instances with multiple interfaces without having terraform rebuild the instances every time.

[masahirotoriumi]

+1

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.