Block/disallow --net=host (host networking) on Mac OS

[vhosakot]

Description

https://docs.docker.com/network/network-tutorial-host/ says:

The host networking driver only works on Linux hosts, and is not supported on Docker for Mac

--net=host (host networking) does not work on Mac as it is not supported on Mac. https://forums.docker.com/t/should-docker-run-net-host-work/14215 also says that --net=host is not supported on Mac.

Hence, --net=host should be blocked/disallowed on Mac and an error like --net=host (host networking) is not supported on Mac should be displayed if --net=host is used on Mac. This way, users on Mac know that --net=host is not supported on Mac and will not waste time debugging host networking issues on Mac.

Steps to reproduce the issue:

1. Run any docker container with --net=host (host networking) on Mac.

   docker run -d --name docker-nginx --net=host -p 80:80 nginx
   

2. Check if the container's port can be accessed on the Mac host.

   $ curl localhost:80
   curl: (7) Failed to connect to localhost port 80: Connection refused
   

Describe the results you received:

Not able to access the container's port on the Mac host when --net=host is used on Mac.

Describe the results you expected:

$ docker run -d --name docker-nginx --net=host -p 80:80 nginx
ERROR: --net=host (host networking) is not supported on Mac.

Additional information you deem important:

Output of docker version:

$ docker version
Client:
 Version:	17.12.0-ce
 API version:	1.35
 Go version:	go1.9.2
 Git commit:	c97c6d6
 Built:	Wed Dec 27 20:03:51 2017
 OS/Arch:	darwin/amd64

Server:
 Engine:
  Version:	17.12.0-ce
  API version:	1.35 (minimum version 1.12)
  Go version:	go1.9.2
  Git commit:	c97c6d6
  Built:	Wed Dec 27 20:12:29 2017
  OS/Arch:	linux/amd64
  Experimental:	true

Output of docker info:

$ docker info
Containers: 1
 Running: 1
 Paused: 0
 Stopped: 0
Images: 1
Server Version: 17.12.0-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host ipvlan macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 89623f28b87a6004d4b785663257362d1658a729
runc version: b2567b37d7b75eb4cf325b77297b140ea686ce8f
init version: 949e6fa
Security Options:
 seccomp
  Profile: default
Kernel Version: 4.9.60-linuxkit-aufs
Operating System: Docker for Mac
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 1.952GiB
Name: linuxkit-025000000001
ID: QWYO:HNLA:VPRN:ZTTG:YUNW:CO37:BWYB:L4Z5:CFDY:JC3H:A6DD:RQDM
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: 29
 Goroutines: 49
 System Time: 2018-03-22T04:19:59.058199778Z
 EventsListeners: 2
HTTP Proxy: docker.for.mac.http.internal:3128
HTTPS Proxy: docker.for.mac.http.internal:3129
Registry: https://index.docker.io/v1/
Labels:
Experimental: true
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

Additional environment details:

Mac OS Sierra (version 10.12.6)

[vhosakot]

Opened this issue per suggestion in moby/moby#36666 (comment).

[pgayvallet]

As you probably know, on D4M, docker is running inside a VM.
So "-net=host is not working" is subject to interpretation : it's working in the fact that the container uses the VM network, it's not in the fact that the VM network is not the host network.

However, in some use case, the actual behaviour is the one expected. That's why we just can't simply disable the whole option on D4D, as it would break for some of our users using it.

The solution is probably more on documentation than on blocking the option.

[vhosakot]

@pgayvallet Thanks for the info. Yes, when --net=host is used on a Mac, good documentation and an example how to access the container's port on the VM network inside the VM on a Mac would be very helpful. The documentation could also include the step to setup port-forwarding on the Mac to the VM's port so that the container's port can be accessed directly from the Mac when --net=host is used.

Also, can you send me steps to access the container's port on the VM network inside the VM on Mac when --net=host is used? I was not able to find them online.

[jakirkham]

Looks related to issue ( #68 ).

[maxthinkthink-tech]

I have spend days for debugging network issues on mac.
And now I found this issue.
It should post some information tell user it's not supported on mac

[docker-robott]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30d of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

[0x1ee7]

/remove-lifecycle stale

[sregger]

As you probably know, on D4M, docker is running inside a VM.

Had no idea this was the case until I started searching for why

docker --net=host

was not working. This is a confusing User Experience and should be improved. Appreciate there is room for interpretation. But common sense usage should win out. I don't think that's currently the case.

Can this issue be re-opened?

[fgarcia]

I always thought that there should be another option like --net=host-virtual or something like that. Then host would work only in Linux and host-virtual in non Linux environments. Failing on non supported option would encourage users to read the manual, not to explore github issues

[deftdawg]

I've probably just wasted 3 days trying to get 2 different applications (home-assistant and mythtv) that will never work without a real implementation of host network because I had guessed given that -p ports are published that the host networking should also work...

Please don't tell me host networking does work but only for the VM, host networking means binding directly to the physical network / physical NIC, the VM doesn't do that and I can't even figure out how to access the VM (docker-machine doesn't seem to do anything on OSX)...

I can't imagine how much time people have collectively wasted troubleshooting this!

Invoking docker with --net=host should fail with an ERROR when started from non-linux OSes.

[vhosakot]

Very happy to see this issue is helping many people! 😄

[deftdawg]

@vhosakot This issue isn't really helping anyone... 😉

But anyway... For those struggling with this problem here's something that will actually help; I've been able to work around 'docker for mac's limitation by the following:

• uninstall 'Docker for Mac'
• install 'Docker Toolbox' for mac
• create the docker machine VM
• run this script which adds a bridged NIC to virtualbox's docker VM (https://raw.githubusercontent.com/blueimp/shell-scripts/df2eaf79ad7935bfb4b6d5834895d4912d6df4ea/bin/docker-machine-bridge.sh)
• capture the IP address by rerunning the script a second time (export CONTAINER_IP=$(docker-machine-bridge.sh))
• start containers with --net=host
• connect to the container at the IP address captured

Since using a Vbox bridged nic puts the docker-machine host VM directly on the network at it's own IP, network=host works from there at it's new IP.

[elver]

This is the most absurdly simple "bug" I've seen where fixing it should take a developer literally minutes, but nobody has done it for over half a year.

[kazzkiq]

Users should at least be warned when trying to use networking in Docker since it's capped in macOS and Windows.

At least a message like:

[Docker Networking]:
We detected you're fiddling with networking, but sadly it's only supported in Linux.

would be enough.

Also, it's 2019. Honestly, buy some macs and PCs and enforce them on your team so they actually see the issues we're having. That would do wonders for QA.

[DonGiulio]

There's indeed some lack of documentation here,

is there a way I can connect to the docker virtual machine so that I could check things before I deploy to prod (which is a Linux system)?

[kojustin]

I just wasted a day of time thinking that this worked the same that it does on Linux.

[vhosakot]

I just wasted a day of time thinking that this worked

@kojustin Wasting just a day on this issue is very productive lol 😉

[mtlott]

And I just wasted 2 hours on this. So I guess I am more productive than the last guy.

But Really???!!???

I don't work with docker day in and out. There should be error or warning message on D4M that reminds about these nuances. My use case is usually picking up a working compose stack and bringing it locally to a Mac to make some tweaks. A little warning that says this one won't work here would have been nice.

[ghost]

And I just wasted 2 hours on this. So I guess I am more productive than the last guy.

But Really???!!???

I don't work with docker day in and out. There should be error or warning message on D4M that reminds about these nuances. My use case is usually picking up a working compose stack and bringing it locally to a Mac to make some tweaks. A little warning that says this one won't work here would have been nice.

same

[danielo515]

Ok, It is very disappointing that this does not work, but I can understand it.
What it is very frustrating is to waste time thinking that the command you just launched works but in reality it does not.
At least a warning is required, please.

[Pasukaru]

I also spent a few hours on this. Once you know why, it makes sense.

However, it should definitely throw an error and not continue without any message whatsoever, making users (including me) think the command will work.

[danperks]

Very excited to hear there's some motion in the ocean on this! Subscribe to the thread, good luck brave devs!

[carlosbaraza]

What a sink of dev time for not implementing a warning, which would take 5 minutes to someone experienced with the docker for mac codebase... I was trapped like so many other people in this thread...

[cocaesprit]

At this point is getting comical

[shatacheruvu]

fixed all my sins all this while but this issue is still open.

[ItsTarik]

Is this a payed feature ? LOOL

[IonasElate]

I would actually pay for this (a solution, not a warning) ;-)

[kopax-polyconseil]

Lucky to learn it now after a few days configuring a platform install. Cannot do as planned, any plan to resolve this ? 😆

[andbi]

During these five years I've turned 50, became a grandfather, survived COVID, and bought a house. Anyone else care to share their stories while the issue is being actively resolved? :-)

[kopax-polyconseil]

During these five years I've turned 50, became a grandfather, survived COVID, and bought a house. Anyone else care to share their stories while the issue is being actively resolved? :-)

It took me a bit less to understand this comment is outdated: #2716 (comment)

I have hit this error: https://forums.virtualbox.org/viewtopic.php?f=8&t=107680

Stderr: 0%...NS_ERROR_FAILURE
VBoxManage: error: Failed to create the host-only adapter
VBoxManage: error: Failed to execute '/Applications/VirtualBox.app/Contents/MacOS/VBoxNetAdpCtl add 2>&1' - exit status: 34304
VBoxManage: error: Details: code NS_ERROR_FAILURE (0x80004005), component HostNetworkInterfaceWrap, interface IHostNetworkInterface
VBoxManage: error: Context: "RTEXITCODE handleCreate(HandlerArg *)" at line 105 of file VBoxManageHostonly.cpp

So there's definitely no way now to run a container with host network ?

[jwoodrow]

The only way I managed to get it to work is to run docker using devcontainers which somehow makes things work out. But this is ridiculous. I understand things aren't simple and you've had to deal with a lot of mac issues what with all the Apple Silicon changes.

But it can't honestly take 5 years to either:

• add an error message on non-linux machines
• Find a solution to make this work on macs

Docker is meant to be a solution to standardize how developers collaborate and run projects on whichever platform they want. But in reality that isn't the case because some options just won't work on other systems and rather than say so directly and reject the command you prefer to have your users rip their hair out of frustration debugging known issues.

[nicks]

Making Host OS networking work on MacOS is tracked here: docker/roadmap#238

Blocking this with an error message is not really on the table, because there are lots of legitimate use-cases for binding to the Host VM network (rather than the Host OS network). For example, Telepresence uses --net=host and wants to bind to the VM network, here's a good post on it: https://www.getambassador.io/kubernetes-learning-center/telepresence-docker-extension

There's been some discussion about making it a warning or having a way to disambiguate like --net=host-os or --net=host-vm, but there are concerns about portability.

[deftdawg]

There should've been a warning when Docker for Mac was first released. At least the command line docker command should warn something about it, even if the Docker for Mac GUI people can't figure out how to handle it.

Besides the 100 or so folks who've found this thread and commented, think of all the other people who give up trying to run a tutorial or a product installation on mac that includes --net=host because it appears to be silently broken.

It would take what to implement a warning? An OS check (docker mac cli), a docker version check (< whatever version will eventually implement it) and a printf?

Blocking this with an error message is not really on the table, because there are lots of legitimate use-cases for binding to the Host VM network (rather than the Host OS network). For example, Telepresence uses --net=host and wants to bind to the VM network, here's a good post on it: https://www.getambassador.io/kubernetes-learning-center/telepresence-docker-extension

There's been some discussion about making it a warning or having a way to disambiguate like --net=host-os or --net=host-vm, but there are concerns about portability.

[disposedtrolley]

cc @davidlopezre

[ryanmccartney]

Happy 5th Birthday to this issue 🍰

[vhosakot]

yay, happy 5th Birthday all! 🍰

[pjamessteven]

Happy birthday for three days ago :( If I had the skill and/or time I would try and fix this, but alas I don't. I hope that one day this issue gets resolved by a majestic computer wizard.

[yrik]

So, what are the workarounds for now?

[bsnk-dev]

If you want docker containers to be able to access a specific application running on say, 127.0.0.1:3000, one solution is to run a proxy inside the container to the host.docker.internal network bridge. You can do so with a utility called socat.

An example so the docker container can access 127.0.0.1:3000 on your machine:

apt-get update && apt-get install -y socat
socat TCP-LISTEN:3000,fork TCP:host.docker.internal:3000

[phiberber]

I'm almost sure the entire team unsubscribed from this Issue, thankfully I got to find out it doesn't work by using Docker Desktop and "lsof" seeing there was no port open, we're near the 6th birthday!

[cristiano182]

@danielo515Ok, obrigado pelo esclarecimento! Existe alguma solução alternativa? Preciso acessar 127.0.0.1da máquina host durante a construção do Docker.

EDITAR: Não importa. Acabei de descobrir que você pode usar http://host.docker.internal:<your-port>para acessar a máquina host em vez de http://127.0.0.1:<your-port>durante a construção.

thank you

[akerouanton]

We added a "host networking" beta feature to Docker Desktop 4.29. You can find more details on this page: https://docs.docker.com/network/drivers/host/#docker-desktop.

Let me close this ticket as 'won't fix'.