Support IPv6

[YtvwlD]

This setup just supports IPv4. This is probably enough for the MVP, but in the future™ we should support IPv6.

Possible solutions:

• letting Traefik run on the host
• https://hub.docker.com/r/robbertkl/ipv6nat/
• https://serverfault.com/questions/276515/use-iptables-to-forward-ipv6-to-ipv4
• https://docs.nginx.com/nginx/admin-guide/load-balancer/tcp-udp-load-balancer/

[YtvwlD]

So, I spent the better part of today's rainy afternoon to do pointless Docker stuff, but I got a working IPv6 connectivity, so yay. Here's what I learned: (the possibilities are sorted from nice to ugly, from difficult to easy)

enable IPv6 in Docker, provide all containers and networks with IPv6

This is a bit of work and not really possible as enable_ipv6 isn't currently supported by Docker Swarm.

Also, this could expose all ports publicly, if done wrong. And I'm no network expert. Also see the next section.

enable IPv6 in Docker, just use it for the ingress network

Docker does support IPv6. And even though the docs are severely lacking details, I tried. And failed.

The Traefik container didn't start because it didn't get an IP address. Perhaps I messed up the subnets (again, I'm no network expert and I don't think I have to to get an HTTP server up and running) or it's because Docker's libnetwork can't correctly calculate network stuff. Also libnetwork's documentation is pretty much non-existant. I'll open an issue there later.

In theory, this should have worked:

echo '{"ipv6": true, "fixed-cidr-v6": "fe80::42:e0ff:fe36:174c/64"}' > /etc/docker/daemon.json
docker network remove ingress
docker network create --driver overlay --ingress --ipv6 --subnet fe80::c000:0:0:0/66 ingress

(see my ipv6 branch for this)

IPv6 NAT

The concept of it sounds horrible, but it would be consistent with the IPv4 setup and I like consistency. And it has the added benefit that this setup actually works.

We need to add robbertkl/ipv6nat/ on the host as a privileged container (!) (outside of Docker Swarm, because it currently doesn't support privileged containers - but it will soon, yay) and modify all services that should be accessible from the outside to not use Docker Swarm's overlay network, but the bridged host network instead - so Traefik:

    - ports:
-      - 80:80
-      - 443:443
+      - target: 80
+        published: 80
+        mode: host
+      - target: 443
+        published: 443
+        mode: host

Also, we need to deactivate the Swarm temporarily to be able to modify the needed bridge network:

docker network create --ipv6 --subnet 172.20.0.0/20 --gateway 172.20.0.1 --gateway fd00:3984:3989::1 --subnet fd00:3984:3989::/64 --opt com.docker.network.bridge.name=
docker_gwbridge --opt com.docker.network.bridge.enable_icc=true --opt com.docker.network.bridge.enable_ip_forwarding=true --opt com.docker.network.bridge.enable_ip_masquera
de=true docker_gwbridge

Also, this effectively disables Swarm's load balancing. We currently just have one node, but yeah.

(see my ipv6nat branch for this)

don't do IPv6 stuff in Docker

This should have been the easiest option. Just let Docker handle the IPv4 traffic and use socat or 6tunnel to redirect IPv6 traffic to IPv4 on the host itself.

But that won't work because Docker already binds to the IPv6 any address - but doesn't handle IPv6 traffic gracefully by default (that's what this whole issue is about). I didn't find an option to disable this.
Normally, you could set the host:port of what an exposed port is bound to, but this just does nothing (but printing a warning) in Swarm mode. So that is that.

We could bind Traefik to a different port and use socat or 6tunnel for both IPv4 and IPv6.

don't do external HTTP in Docker

We could let Traefik run on the host and not inside a container. This would solve all of this magically. But it complicates management of Traefik itself.

don't do IPv6 on this VM

We could point the AAAA record to another VM which proxies the traffic to this VM over IPv4. But that would break SSH.

[joergmschulz]

if you don't do IPv6 in docker (which seems to be the best idea), you can always use haproxy and proxy the ports in question to ipv4.
For some cases, this might be undisireable: when you need the original source IP address.
As the situation hasn't changed much on docker side, I hope this is not necroposting....

[YtvwlD]

As the situation hasn't changed much on docker side, I hope this is not necroposting....

No, you're good. This is still unsolved for us. :)

But Docker has gained a somewhat-functional IPv6 support which could even be enabled by default in the future, see robbertkl/docker-ipv6nat#65 for details. Currently, it is behind the experimental flag and may have some rough edges, so I figured it's too early for us.

if you don't do IPv6 in docker (which seems to be the best idea), you can always use haproxy and proxy the ports in question to ipv4.

In theory, yes, in practice this didn't work in the past because Docker still binds to IPv6, so haproxy can't start (see the comment above).
With the new IPv6 support in Docker present but disabled, it may just bind to IPv4 by default but I'm not sure.

[joergmschulz]

It is as you described. By default, especially in swarm mode, IPv4 is deactivated by default. So, just yesterday, using current versions, I did it the way I described: the ports needing V6 support can easily be grabbed by haproxy. For imap , excluding 80,443, ... which are used by nginx in my setup, it looks like:

global
        log /dev/log    local0
        log /dev/log    local1 notice
        chroot /var/lib/haproxy
        stats socket /run/haproxy/admin.sock mode 660 level admin
        stats timeout 30s
        user haproxy
        group haproxy
        daemon

        # Default SSL material locations (....)

defaults
        log     global
        mode    tcp
        # option        httplog
        option  dontlognull
        timeout connect 5000
        timeout client  50000
        timeout server  50000
        # errorfile 400 ... 

frontend imap.blub.bla
       bind 2001:CAFE:CAFE:CAFE::CAFE:144
       default_backend imap_servers

backend imap_servers
      server imap 127.0.0.1:143
# repeat front/backend for every port

[YtvwlD]

I've tried the experimental ip6tables support in Docker and it doesn't really work.
Nothing happens automatically, but by manually configuring the docker_gwbridge network, containers get an IPv6 address and might be able to reach the outside world. (I haven't tested this, though.)

But that doesn't work for the ingress network: I can create the network, but then Traefik doesn't start anymore with "Invalid address fd00:3984:3990::2: It does not belong to any of this network's subnets", even though I'm pretty sure it does.

But mode: host surprisingly works even without enabling an extra option. Putting

    ports:
      - published: 80
        target: 80
        mode: host
      - published: 443
        target: 443
        mode: host

in the traefik.yml makes Traefik reachable via IPv6.