Support Host Network (--network host) mode on Docker Desktop for Mac and Windows

[hexcowboy]

Tell us about your request

Right now, docker run --network host <image> is only available to Linux docker hosts. From the official documentation:

The host networking driver only works on Linux hosts, and is not supported on Docker Desktop for Mac, Docker Desktop for Windows, or Docker EE for Windows Server.

Which service(s) is this request for?

Docker Desktop for Mac, Docker Desktop for Windows

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?

Making a docker container with dynamic port mappings is not possible right now.

For example, a container that allows a user to accept connections on arbitrary ports through a netcat listener. During a network penetration test, the user may be testing outbound firewall rules and needs to dynamically accept connections on different ports in the container, eg 53, 80, 443, 1337, 9001, or any other arbitary port. The user will not know these ports before starting the container, so it's not possible for the user to run the container with docker run -p <ports> <image>.

With docker run --network=host, this could be accomplished by just using the same network schema as localhost.

Are you currently working around the issue?

The "best" workaround I know of would to add an EXPOSE 1-1000 in the Dockerfile and run the container with docker run -P <image>, which is very slow and does still limits the range of open ports. Using the full port range of EXPOSE 1-65535 hangs my terminal, so I don't know how effective it is.

The -P flag is documented here: https://docs.docker.com/network/links/#connect-using-network-port-mapping

This is not a sustainable workaround.

Additional context

• https://stackoverflow.com/questions/49323225/expose-all-ports-for-a-docker-image/49323975
• https://stackoverflow.com/questions/54165483/docker-alternative-to-network-host-on-macos-and-windows

[Amondale]

Although documentation stats not supported on any O/S but Linux, it also doesn't seem to work correctly (although it doesn't throw error) on Linux flavors on WSL2. The swarm configurations likewise don't function as they do in "bare metal" Linux, which can make for a ton of confusion in the docs.

There is a large and growing population of Mac/Windows users that are not fully supported by the extant documentation, and issues like this (many other networking issues exist on those platforms too).

I plan to follow this thread closely, and would welcome the chance to help refine docs so they apply across the board, or exceptions are noted where they exist.

[gramss]

What is basically needed to make this work? Where would one need to start looking to make this work on MacOS?
Any basic information on how to start that would be much appreciated.
I think technically this should be doable, also from the community. It's currently just not on the roadmap..?

[zfil]

Having this supported could maybe help with the horrible network performance I see on macos docker ...

This makes for instance using postgres in a container very painful.
Running our test suite using a postgres in a container and with docker 4.6:
virtualization.framework: 16min
hypervisor.framework: 10min
locally installed postgres: 3min

[christophermclellan]

Hi - thanks for this issue and sorry for the delayed reply. We'll do some scoping on this and circle back on this thread shortly.

[wiredwiz]

This is certainly a feature I would love to see support for. lacking host network support makes it impossible for me to run services that need to be network aware on anything other than linux, since otherwise they only see the network bridge.

[hexcowboy]

Another limitation I've found - not being able to run a local Redis cluster https://redis.io/docs/manual/scaling/#redis-cluster-and-docker

[ReubenFrankel]

Related to docker/desktop-linux#87

[gramss]

Now with the new release 4.14.0 of Docker Desktop on Mac utilizing the Apple Virtualization Framework, it should be possible to add another network adapter featuring the bridged network aka. network_mode = host ?

https://developer.apple.com/documentation/virtualization/vzvirtualmachineconfiguration/3656724-networkdevices

Is this a big challenge? Can somebody point out the hypervisor configuration for the Apple Virtualization Framework here? :)

Also thanks for putting it in the "Considering" space of the docker-roadmap ! 👍

[logical-and]

+1 - would be really nice to get this implemented as it works on Linux!

[jwoodrow]

Now with the new release 4.14.0 of Docker Desktop on Mac utilizing the Apple Virtualization Framework, it should be possible to add another network adapter featuring the bridged network aka. network_mode = host ?

developer.apple.com/documentation/virtualization/vzvirtualmachineconfiguration/3656724-networkdevices

Is this a big challenge? Can somebody point out the hypervisor configuration for the Apple Virtualization Framework here? :)

Also thanks for putting it in the "Considering" space of the docker-roadmap ! 👍

Is there any way to get more updates on this ? I think this would be an extremely appreciated feature for mac users

[davidthornton]

Here's the link to the filter to track its progress!

https://github.com/docker/roadmap/projects/1?card_filter_query=network+host

[dewdrinker19]

Yes please. Very much please.

[ghost]

Here's the link to the filter to track its progress!

https://github.com/docker/roadmap/projects/1?card_filter_query=network+host

It doesn't seem to be getting traction or is it too early to tell?

[alikhanich]

I've been waiting for 5 years for that. Can you please do it? If the security is main concern please disable this feature by default.

[BlaiseOfGlory]

bump. I have several tools that I would love to be able to containerize that need layer2 access.

[ShutdownRepo]

For everyone's bummed about this lack, the solution may very well be to switch to a more active project 🤷 It's been five years. For instance, https://orbstack.dev/, while not open-source, seems very promising and already supporting the --network=host mode (among other things). I'm amazed to see independent projects like this one surpass Docker Desktop (for mac in this case) performance and feature-wise.

[ghost]

https://orbstack.dev/

Website has been registered for only 1 week. How well do you know it?

[ShutdownRepo]

https://orbstack.dev/

Website has been registered for only 1 week. How well do you know it?

Not well myself, but I work with 2 people that tested it and that raised excellent feedback.

[bryansh]

Good news! This has been added as a beta feature in Docker Desktop 4.29.

[miamilabs]

Good news! This has been added as a beta feature in Docker Desktop 4.29.

yea its amazing for some features. I managed to run home assistant now outside of VM. But faced now new issues with USB stuff which cant be mouted to docker since 6 years as well :)

Good Luck all. Hope people can resolve problems with this update

[Nemesis7]

Just downloaded and installed Docker Desktop on my Mac to test this out with the Scrypted docker-compose file in network_mode: host but I can't seem to connect using localhost, it does work when I use the eth1 ip-address of the docker container. Am I missing something? And maybe because of that I cannot add a HomeKit device to my setup (mdns issue?)

[ca0x]

Yeah, I also tried with scrypted and homebridge. It's not working as intended. Although it works with nginx.

Here are the open ports in the VM and on the host with both nginx and homebridge when using network_mode: host

nginx

nginx (Docker VM)
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN  
tcp        0      0 :::80                   :::*                    LISTEN 

nginx (host)
com.docke  879            user  238u  IPv6 0x1234...      0t0    TCP *:80 (LISTEN)

homebridge

homebridge (Docker VM)

tcp        0      0 :::8581                 :::*                    LISTEN
tcp        0      0 :::51826                :::*                    LISTEN
udp        0      0 0.0.0.0:62891           0.0.0.0:*

homebridge (host)

com.docke  879            user  289u  IPv6 0x1234...     0t0    UDP *:62891

It seems that only the ports listening on IPv4 addresses are exposed.

[vadimen]

C:\Users$env:USERNAME\AppData\Roaming\Docker\settings.json

"hostNetworkingEnabled": true,

Version Docker Desktop 4.29.0 Settings ⇒ Features in development ⇒ Enable host networking Host networking allows containers that are started with --net=host to use localhost to connect to TCP and UDP services on the host. It will automatically allow software on the host to use localhost to connect to TCP and UDP services in the container. Sign in required.

that is very cool, can we activate it using powershell on windows server core ?

Try my script:

# Read the JSON file
$path = 'C:\Users\{0}\AppData\Roaming\Docker\settings.json' -f $env:USERNAME
$json = Get-Content -Raw -Path $path | ConvertFrom-Json

# Check if the property exists
if ($json.PSObject.Properties.Name -notcontains 'hostNetworkingEnabled') {
    # Add the property if it does not exist
    $json | Add-Member -Type NoteProperty -Name hostNetworkingEnabled -Value $true
} else {
    # Change the value
    $json.hostNetworkingEnabled = $true
}

# Save the changes back to the file
$json | ConvertTo-Json -Depth 100 | Set-Content -Path $path

I have both "hostNetworkingEnabled": true and "allowExperimentalFeatures": true. Stopped and started docker service. When I run "docker run --network host hello-world" I am getting "docker: Error response from daemon: network host not found." .

[CorentinDev]

Awesome, this update solved my issue! Thank you so much! 🤩

[ca0x]

Yeah, I also tried with scrypted and homebridge. It's not working as intended. Although it works with nginx.

Here are the open ports in the VM and on the host with both nginx and homebridge when using network_mode: host

nginx

nginx (Docker VM)
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN  
tcp        0      0 :::80                   :::*                    LISTEN 

nginx (host)
com.docke  879            user  238u  IPv6 0x1234...      0t0    TCP *:80 (LISTEN)

homebridge

homebridge (Docker VM)

tcp        0      0 :::8581                 :::*                    LISTEN
tcp        0      0 :::51826                :::*                    LISTEN
udp        0      0 0.0.0.0:62891           0.0.0.0:*

homebridge (host)

com.docke  879            user  289u  IPv6 0x1234...     0t0    UDP *:62891

It seems that only the ports listening on IPv4 addresses are exposed.

Disabling IPv6 in the VM solved the ports issue, exposing all ports and allowing network access to the Homebridge UI. However, it appears that mDNS is not functioning, and the service remains undiscoverable by HomeKit.

Access VM

docker run -it --rm --privileged --pid=host justincormack/nsenter1

Disable IPv6

sysctl -w net.ipv6.conf.all.disable_ipv6=1

[akerouanton]

@ca0x Thanks for reporting. It seems you're hitting two limitations:

Software working in dual-stack mode, or IPv6-only can't benefit from this improvement. As we're working toward adding better support for IPv6, we're going to fix this mishap soon -- most probably in v4.31.

mDNS is currently not supported. We need to investigate exactly why.

@vadimen Are you trying to use this feature with Windows Containers? This improvement works only if there's a Linux VM running.

@Nemesis7 What ports Scrypted is listening to? Are some of them bound by DD on the host? It's probably worth filling a new issue in either https://github.com/docker/for-win or https://github.com/docker/for-mac.

[vadimen]

@ca0x Thanks for reporting. It seems you're hitting two limitations:

Software working in dual-stack mode, or IPv6-only can't benefit from this improvement. As we're working toward adding better support for IPv6, we're going to fix this mishap soon -- most probably in v4.31.

mDNS is currently not supported. We need to investigate exactly why.

@vadimen Are you trying to use this feature with Windows Containers? This improvement works only if there's a Linux VM running.

@Nemesis7 What ports Scrypted is listening to? Are some of them bound by DD on the host? It's probably worth filling a new issue in either https://github.com/docker/for-win or https://github.com/docker/for-mac.

Yes, I was trying to use it with windows containers. Could you please suggest me any solution for this use case https://devops.stackexchange.com/questions/19117/bind-dedicated-ip-to-docker-container-in-windows-server-2022 ?

[akerouanton]

@vadimen I'm not super familiar with Windows Containers networking. I think it's worth looking for a similar issue or asking on https://github.com/microsoft/Windows-Containers.

[Nemesis7]

@Nemesis7 What ports Scrypted is listening to? Are some of them bound by DD on the host? It's probably worth filling a new issue in either https://github.com/docker/for-win or https://github.com/docker/for-mac.

@akerouanton By default it's port 10443, plus there one port for HomeKit which is variable and there is the mDNS port i.e. 5353, but shouldn't this recent release allow any container with network_mode: host to be reachable via localhost? For Scrypted I can create a new issue, but I think reachability via localhost is related to this thread, no ?

[akerouanton]

@Nemesis7 It's definitely related but it'd be better to have a proper bug report.

[AndreKR]

Hm, am I missing something?

I'm running 4.30.0:

Under "Features in development" I enabled "Enable host networking":

(For unknown reasons that checkbox only became enabled when I logged into an account, it's unclear to me how networking and login are related.)

If I now run netcat in listen mode in a container:

I am unable to connect to that from outside the container:

But if I exec into the container and try the connection from there, it works:

That should work from the host as well, or not?

[tangledhelix]

Using -l and -p together is an error. From nc(1):

     -l      Used to specify that nc should listen for an incoming connection
             rather than initiate a connection to a remote host.  It is an
             error to use this option in conjunction with the -p, -s, or -z
             options.  Additionally, any timeouts specified with the -w option
             are ignored.

     -p source_port
             Specifies the source port nc should use, subject to privilege
             restrictions and availability.  It is an error to use this option
             in conjunction with the -l option.

Also, when using listener mode the port is random:

/ # nc -lv
listening on [::]:56323 ...

If you run a listener like this, and then try to connect to it using localhost on the outer system, it still won't work, because you can see it is bound to [::] which is the IPv6 stack. The --net=host parameter will only bind to IPv4, so the last piece of the puzzle is to specify an IPv4 address to bind to...

/ # nc -lv 0.0.0.0
listening on 0.0.0.0:59271 ...

And after this, connecting to localhost on the outer host, using the port in the message (59271 in this example) should work. I tested this on Mac, although I see you're using Windows. I would hope it works the same, but YMMV.

[AndreKR]

What image are you using for your container? The netcat syntax you're using is for BSD netcat, for GNU and Busybox netcat my syntax should be ok.

I tried installing BSD netcat in my Alpine container but I couldn't get it to work with your commands:

I tried specifying a local IPv4 address to rule out the IPv6 issue you mention with nc -l -p 1234 -s 0.0.0.0 and nc -l -p 1234 -s 127.0.0.1 but the netcat is still not reachable from the host.

[tangledhelix]

Based on your screenshot, to replicate your environment, I used alpine:latest.

❯ docker image ls alpine:latest
REPOSITORY   TAG       IMAGE ID       CREATED       SIZE
alpine       latest    8e1d7573f448   2 years ago   5.33MB

I will admit I probably used the manpage from macOS (which would have the BSD version), not realizing the disparity between it and GNU/Linux. But the behavior I tested and pasted output from came from alpine:latest:

❯ docker run -it --rm --net=host alpine:latest
/ # nc -lv 0.0.0.0
listening on 0.0.0.0:55201 ...

I see now that your syntax works and has the right port (which it clarifies with -v). But, it binds to IPv6 only, which leads back to my observation about needing to use IPv4:

/ # nc -l -p 1234 -v
listening on [::]:1234 ...

If you bind to the IPv4 stack it then allows connection from the outside host:

/ # nc -lk -p 1234 -v -s 0.0.0.0 -e date
listening on 0.0.0.0:1234 ...
connect to 192.168.65.3:1234 from 192.168.65.1:38201 (192.168.65.1:38201)

❯ nc localhost 1234
Sat May 18 22:32:06 UTC 2024

[AndreKR]

Using your command line doesn't work for me:

[tangledhelix]

Ah, sorry to see that. Unfortunately I have no explanation for it. I would guess it is a difference in the Mac vs. Windows environment (either the network stack, or the way Docker Desktop works with the host OS, or both). I don't have any Windows machines around here to test it.

[s13n]

It doesn't work with me. I'm using v4.33.0 on Windows 11, and have enabled the feature after signing in. Alas, docker info doesn't show the host network mode as available, and starting a container with --net=host throws an error.

Network: ics internal l2bridge l2tunnel nat null overlay private transparent

Am I doing something wrong? Do I have to create the host network somehow?

[akerouanton]

@s13n This feature isn't compatible with Windows containers. It's meant to be used with Linux containers, on both macOS and Windows.

[s13n]

@akerouanton Thanks! That wasn't clear to me, and the explanatory text near the tick box in the settings dialog doesn't mention it. Maybe it would be an idea to disable the choice when Docker Desktop is in Windows container mode.

[akerouanton]

@s13n Thanks for reporting! This should be fixed in the next release (4.34).