network_mode: "host" probably not working as expected

[ealves-pt]

I'm trying to run from a docker-compose.yml the a service with network_mode: "host" but it doesn't seem to be working as expected in OS X.

Having the docker-compose.yml:

version: '2'
services:
  db:
    image: postgres:9.6.1-alpine
    environment:
      POSTGRES_USER: arexdb_dev
      POSTGRES_PASSWORD: arexdb_dev
      PGDATA: /var/lib/postgresql/data/pgdata
    volumes:
      - ~/docker-data/postgresql:/var/lib/postgresql/data/pgdata
    expose:
      - "5432"
    ports:
      - "5432:5432"
    restart: always
    network_mode: "host"

Expected behavior

Running nmap -sT 127.0.0.1 should output:

PORT     STATE SERVICE
5432/tcp open  postgresql

Actual behavior

Running nmap -sT 127.0.0.1 doesn't show the expected port.

[djs55]

Unfortunately Docker for Desktop doesn't currently support the "host" network_mode where containers are able to freely bind host ports without being managed by docker. Instead, ports must be explicitly whitelisted in the docker run or the docker-compose.yml.

I notice that you have white-listed port 5432 in your docker-compose.yml. If you use the "bridge" network mode, e.g.:

    network_mode: "bridge"

then port 5432 should be bound on the host.

$ netstat -an | grep 5432
tcp6       0      0  ::1.5432               *.*                    LISTEN     
tcp4       0      0  *.5432                 *.*                    LISTEN  

I believe "bridge" mode is actually the default so you could delete the network_mode line from your docker-compose.yml if you wished.

In the longer term we're still considering what it would take to make network_mode: host work. If you have particular use-cases in mind, please let us know.

[arup42]

There are plenty more issues related to net=host not working the way it does on linux, like #155 (which has the most useful information on it) and maybe #68 and #57

If this is going to be where this issue is consolidated, please also take note of this thread that sounded like it might contain a workable solution but hasn't been commented on either way by someone from docker as far as I know.

As far as use cases, I use docker to run an application that (1) scans the local network looking for other devices and needs to see their mac addresses, and (2) dynamically binds to a local port that will later be used by other hosts to contact it. Both of these things work fine on linux with --net=host, and don't work as expected on a mac.

Apologies in advance if I'm conflating things that you guys see as distinct issues, but it would definitely help if someone from docker could provide some guidance on whether it's realistic to hope that --net=host will ever work the same way it does on linux. Thanks.

[ingshtrom]

I'll add our current use case that is failing right now:

We have nginx acting as a reverse proxy for our other services. Each service and nginx are all in separate containers. We run our services in Swarm mode with 1.13 (using docker compose v3), and then using another compose file, run nginx, which is supposed to communicate with each service. We could add nginx to the swarm or use container linking, but then we have to change our nginx config in production vs local dev because we run nginx directly on hosts in production. I did get that working with a sed command at startup and passing in the host IP as an environment variable, but then there is the problem of the Mac changing IPs, so every time Mac decides to change IPs, the developer needs to re-setup the whole environment...

The idea was that we could run nginx in network_mode: 'host' and then always reference 127.0.0.1:<service_port>, which is exactly the same way we do it in production. Unfortunately, when I set that up, I only get empty responses from hitting port 80 or 443 locally when nginx is started up (aka what the OP said).

I am open to workarounds or other ideas, but host mode would make this a lot easier :)

[manfredriem]

It appears it is possible for a standard xhyve install is able to allow access to the xhyve VM from the outside as indicated by the blog post at http://mifo.sk/post/xhyve-for-development

I think this would be the first step to properly supporting --net=host in Docker for Mac

Can someone from the Docker team investigate this?

[rahul-mali-mpf]

ERROR: The Compose file './docker-compose.yml' is invalid because:
Unsupported config option for segmentcreationservice: 'network_mode'
I am running on docker-compose version 1.8.0

[sam3d]

I'd just like to add my use-case as it's pretty frustrating.

I have a media server setup on my system where I host about five different services to support my media server setup (Jackett, Plex, Radarr, Sonarr, Transmission) and I only ever need to run all or none of these, and I'd like the individual applications to be isolated from the host (so this is a perfect use case for docker-compose).

Two of the five services have it recommended that you bind the services directly to the "host". Also, unlike on linux, Plex will show the following on Mac OS X after having to manually bind the 2 TCP ports and 4 UDP ports:

And is not at all fun to use in this mode (with the incorrect hostname and an "indirect" connection).

[otherguy]

It would also be very useful for e.g. having npm run dockerized. Currently I have an alias set alias npm='docker run --rm --interactive --tty --workdir /opt -v $(pwd):/opt node:latest npm' but anything that binds a port, like ng needs to be run manually with port binding. Just being able to run npm run serve would be nicer.

[apahne]

Are there any news on this? Do you think that this issue will be addressed anytime soon?

[barakbd]

I would like to connect to my local mongodb on the host machine from inside my container, since mongodb cannot be mirrored inside a container using mapped volumes, and I would like to continue working with the same db when developing locally or containerized. Also, in prod we are using a managed service of mongodb and not a local mongo container.
https://docs.docker.com/samples/library/mongo/#where-to-store-data

WARNING (Windows & OS X): The default Docker setup on Windows and OS X uses a VirtualBox VM to host the Docker daemon. Unfortunately, the mechanism VirtualBox uses to share folders between the host system and the Docker container is not compatible with the memory mapped files used by MongoDB (see vbox bug, docs.mongodb.org and related jira.mongodb.org bug). This means that it is not possible to run a MongoDB container with the data directory mapped to the host.

Docker version 18.03.0-ce, build 0520e24
docker-compose version 1.20.1, build 5d8c71b

I was hoping to use netowrk_mode: "host" in my docker-compose.yaml, however it doesn't seem to work.
Any suggestions?

[YRM64]

Ladies and Gents,

I believe the jury is still out on the issue of supporting --net=Host. At this point, I wouldn't even recommend a workaround because too much of the information available is dated. However, developers, I found a Web-site, and a blog post at http://www.forum.synology.com., the post is dated January 15, 2017, 6:52 pm. The post is by an individual identified as 'mightbetrue'. Mightbetrue says, "Just wanted to say that Docker in DSM6 is able to handle NET=HOST using the checkbox at the bottom of the network-tab during the creation of a container". "The setting is not available for existing containers". There is no reason to tinker around in any .conf file anymore".

I don't have an operating system for testing, so I advise everyone wanting to proceed, to proceed with caution! And Good-luck!

[otherguy]

DSM 6 us the Synology operating system, which is based on Linux.

This thread is about Docker for Mac!

[risa]

I understand that there may be limitations for Docker on Mac. After all, it is still a free product, and I am grateful to the developers for such a great tool.

However when a feature is not supported on one platform, PLEASE ISSUE A WARNING OR EVEN BETTER STOP WITH AN ERROR!

I just spent a few hours trying to debug deployment of development stack that worked on Linux some time ago a tearing my hair out why things do not work. And as usually I first tried hard to find the bug in my own code.

[andreypp]

+1

[buckfullingham]

Like many others, I followed the doco when attempting to get eclipse to talk to my local xquartz server on my mac and it doesn't work.

Please also note that along my travels I found that unix sockets also aren't properly shared between container and host when mapped using run -v.

[docker-robott]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30d of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

[docker-robott]

Issues go stale after 90 days of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

[nick-zh]

/lifecycle frozen

[MCFreddie777]

What's the status on this task?

I have a service running on my mac, and I need to connect to it from inside of container.
Is there any solution other than rewriting all localhost to host.docker.internal everywhere in the code?

[dhirajpatra]

This problem with Docker desktop Mac v 20+ downgrade to compile with network mode host. As it is required for ECS Farget.

[folex]

Docker Desktop: 3.3.3 (64133)

Services inside containers do not bind to the host networks interface.

[vordimous]

I found a two solutions to my problem that worked.

First in this Stackoverflow article. I hope this helps someone.

    entrypoint:
      - "sh"
      - "-c"
      - |
          echo "$$(getent hosts host.docker.internal | awk '{ print $$1 }') localhost.com" >> /etc/hosts;
          cat /etc/hosts;

Output from the entrypoint:

tomcat-apps_1  | 127.0.0.1      localhost
tomcat-apps_1  | ::1    localhost ip6-localhost ip6-loopback
tomcat-apps_1  | fe00::0        ip6-localnet
tomcat-apps_1  | ff00::0        ip6-mcastprefix
tomcat-apps_1  | ff02::1        ip6-allnodes
tomcat-apps_1  | ff02::2        ip6-allrouters
tomcat-apps_1  | 172.29.0.4     b4e48b375e5a
tomcat-apps_1  | 192.168.65.2 localhost.com

Second in the docs network section

    networks:
      default:
          aliases: 
              - localhost.com

[tugot17]

@vordimous and after adding the this two sections you were able to request the localhost?

[vordimous]

@vordimous and after adding the this two sections localhost you were able to request the localhost?

You still can't use the localhost keyword. but if the app trying to call it has a problem calling host.docker.internal then this entrypoint script did let me point to the alias.

[sharq88]

Is there any solution other than rewriting all localhost to host.docker.internal everywhere in the code?

Thought to mention.. I had a similar issue, what I've done is created a separate .dockerfile for mac which had an extra:

RUN sed -i 's/localhost/host.docker.internal/g' /opt/app/config/development.json

And pulled all localhost entries to the config file. This allowed overwriting seamlessly only for mac. Not perfect, but it was okay-ish until a better option comes out.

[sfratini]

I'd like to add another use case, albeit a niche one. I am trying to dockerize our dev environments as much as possible. One of those containers is a flutter container where we pass our commands. We are currently trying to start the app and connect to a local simulator (iOS Simulator to run the app), however the setup still is not working. With network host we could probably make flutter find the devices locally and work much easily.

[hybras]

As far as use cases, I use docker to run an application that (1) scans the local network looking for other devices and needs to see their mac addresses ...

Would like to add a use case of mine that fits under this category. I am running pihole DNS within a container. Using a bridge network allows basic functionality. However, I'd like pihole to see hosts on my network so I can see per device stats and use per device blocking.

[gabo-magnet]

Now with the new release 4.14.0 of Docker Desktop on Mac utilizing the Apple Virtualization Framework, it should be possible to add another network adapter featuring the bridged network aka. network_mode = host ?

https://developer.apple.com/documentation/virtualization/vzvirtualmachineconfiguration/3656724-networkdevices

Is this a big challenge? Can somebody point out the hypervisor configuration for the Apple Virtualization Framework here? :)

Also thanks for putting it in the "Considering" space of the docker-roadmap ! 👍

docker/roadmap#238 (comment)

[akerouanton]

Host networking reached GA in Docker Desktop v4.34, and we removed the sign-in requirement in v4.35. So let me close this ticket.