Bug: iptables rules setup to intercept ingress traffic do not intercept all traffic

[kiranmeduri]

Summary
What are you observing that doesn't seem right?
iptables rules on the ingress do not intercept all traffic. Currently they only intercept traffic destined to app-port and forward that traffic to Envoy. This has the following side-effects

1. Traffic internal to the application e.g. sidecar statsd, x-ray, that need to be only accessible on localhost are accessible from outside, even when there is no corresponding listener is configured. Though this enables some use-cases around multiple listeners (currently not supported by App Mesh, Feature Request: Add support for multiple listeners on Virtual Node and Virtual Router #120 ), it deviates from the expectation that only opt-in (explicit) listeners are accessible from outside.
2. Makes it harder to debug if traffic to app-port is actually flowing through Envoy when the configuration on application does not match virtual-node listener. Users may think the traffic is being intercepted by Envoy but actually it is side-stepped.

It is recommended practice to bind all the non-Envoy ports to localhost (127.0.0.1) to avoid inadvertent access, but many applications on-boarding onto service mesh do not make that change.

Steps to Reproduce
Create an ECS task or Kubernetes pod with two containers that listen on different ports (or same container listening on different ports). Mark one port as app-port. Now try to access the application on another port and you should see that it works. Expectation is to get 404.

Are you currently working around this issue?
How are you currently solving this problem?

Additional context
Anything else we should know?

CNI: https://github.com/aws/amazon-vpc-cni-plugins/blob/master/plugins/aws-appmesh/plugin/commands.go#L143

Attachments
If you think you might have additional information that you'd like to include via an attachment, please do - we'll take a look. (Remember to remove any personally-identifiable information.)