Use ENI & EIP to get closer to HA for NAT egress instances

[emmanuel]

As described here, attaching an Elastic IP (EIP) to an Elastic Network Interfaces (ENI) provides a foundation for building HA NAT.

[emmanuel]

Currently we run a script on boot on each NAT instance, which assigns the default route for the paired private subnet's route table to the booting NAT instance. The instance is assigned an IAM role that allows modifying the route table of the private subnet.

In the ENI+EIP model, instead of updating the private subnet's route table, the route table's default route is statically configured to direct traffic to the ENI. The NAT instance assigns the appropriate ENI to itself on boot.

The EIP is a cherry on top that presents a stable return address to the outside world. That facilitates the eventual goal of 2 NAT nodes per private subnet in a hot-standby scenario. Hot-standby remains for future work.

[emmanuel]

One stumbling point is that terraform doesn't currently support disabling source/destination checks on the ENI. It looks like this just landed in terraform, but hasn't seen a release yet.

[pidah]

disabling source/destination checks on the ENI has been added in the 0.6.2 release

[emmanuel]

True that. Time to close this!