Enable ECS with other networking modes

[jamsajones]

In ECS awsvpc networking mode doesn't meet everyone's needs. Enabling other network modes would be great.

[jlambert121]

We have been a long time ECS user (pre-awsvpc) and unfortunately the way our environment is built we cannot easily switch. Once bridge mode is supported we will be able to do a larger production rollout of this!

[idubinskiy]

We would love to be able to switch to awsvpc networking, but many of the services we run on ECS use small enough allocations of CPU and Memory that binpacking places significantly more tasks per instance than the limit of ENIs that can be attached to each instance. Either EC2 needs to support a lot more ENIs per instance, or App Mesh needs to support bridge mode.

[jamsajones]

@jlambert121 thanks for the input!

[jamsajones]

@idubinskiy thanks for the input! Is it just ENI density that keeps you from moving to awsvpc or are there other reasons to stay with bridge?

[idubinskiy]

@jamsajones We also use Ubuntu as the base for our ECS instances, so that's not officially supported, but it seems that there are ways to get awsvpc working on it. However, we haven't made an attempt because of the ENI density, so there may be other issues that could come up.

[rlafferty]

To echo what @idubinskiy said, the ENI limit is definitely very restrictive for us to even consider a move to using awsvpc task networking in our environment. Our typical ECS container instance usually has 15+ tasks running.

[joshuabaird]

@jamsajones We have the same issue(s) as @idubinskiy. It's not uncommon for us to have 15-20 containers on our instances (m5.2xlarge). To support that number of containers, we would need to switch to at least an m5.24x (which still only supports 15 ENIs). Not to mention the amount of CPU/memory we would be wasting on such a large instance when we're only able to pack 15 containers on it. Surely this is a concern for other customers as well?

[jamsajones]

@idubinskiy thanks for the info re:Ubuntu and awsvpc. The I will reach out to the ECS Agent team and have them look into that issue.

[jamsajones]

For the ENI density topic, this is a common request from our customers and is now part of the public roadmap for the AWS Containers Org. Feel free to comment over there if you have not yet aws/containers-roadmap#7.

From an App Mesh perspective we are still going to look into ways to support other modes as awsvpc may not fit everyone's needs.

[idubinskiy]

@jamsajones The roadmap is really helpful. However, as far as commenting, the "issues" all have this message: An owner of this repository has limited the ability to comment to users that are collaborators on this repository.

Edit: It's also not possible to create an issue in that repo right now, despite the README there saying

Q: How can I provide feedback or ask for more information?

A: Please open an issue!

[abby-fuller]

Hey @idubinskiy! Containers roadmap should be open for comments and issues as of yesterday. There was a brief window when we went public and it got popular that we hit a setting and it limited it to repository owners. All is open now! Def feel free to come open issues/comment.

[idubinskiy]

@abby-fuller Thanks for the update!

[coultn]

Hi everyone, FYI we have an upcoming launch for ECS that will increase the number of ENIs on ECS when using awsvpc networking mode on nitro-based instances. This will help make it easier to use App Mesh with awsvpc networking on ECS: aws/containers-roadmap#7

[shubharao]

We now have increased task limits when using awsvpc task networking mode. This let you go up to 120 tasks per instance, depending on instance type. More details here and here
Based on this, we do not have plans to support other networking modes in the near term roadmap. If this is a concern for you, please open a new issue. Thanks!