Binding to a single ip

[NoUseFreak]

Describe the bug
I am trying to bind the full cluster to a single ip. I have mulitple nics on my server and would like to run it as a single node cluster.

Can someone help me get the list of arguments to run it on a single ip only.
Currenlty once it starts, it seems to bind port 80/443 on every ip and responding with a 404 not found.

I already tried setting the following:

./k3s server --kube-proxy-arg="bind-address 192.168.123.100"

[warmchang]

try ./k3s server --bind-address=192.168.123.100

https://github.com/rancher/k3s/blob/0e3711b8b709a06a3769e89fe7e3ef54c48a0cd0/pkg/cli/cmds/server.go#L38-L40

[NoUseFreak]

@warmchang Looks like it still binds to more than just the provided ip.

[ibuildthecloud]

@NoUseFreak The argument should have = in it, ./k3s server --kube-proxy-arg="bind-address=192.168.123.100" Are you sure that is really the setting you want though? I'm not even totally create what that address is used for. Most of the time people want to set --nodeport-addresses which controls the ip ranges uses in the iptables.

[davinkevin]

I have the same problem, every network inteface see ports 80/443 associated to k3s 😓.

Even if I use the --flannel-iface ifname...

[stale[bot]]

This repository uses a bot to automatically label issues which have not had any activity (commit/comment/label) for 180 days. This helps us manage the community issues better. If the issue is still relevant, please add a comment to the issue so the bot can remove the label and we know it is still valid. If it is no longer relevant (or possibly fixed in the latest release), the bot will automatically close the issue in 14 days. Thank you for your contributions.

[davinkevin]

I miss the notification… someone can reopen it please?

[cjellick]

@davinkevin sure, but can you provide a more complete report on what your experiencing and how to reproduce, including the version of k3s that you're running.

[davinkevin]

With a complete new install on a VM with nothing except k3s, I have 2 network interfaces:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp0s5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:1c:42:5f:e1:c9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.228.95/16 brd 192.168.255.255 scope global dynamic enp0s5
       valid_lft 85656sec preferred_lft 85656sec
    inet6 fe80::21c:42ff:fe5f:e1c9/64 scope link
       valid_lft forever preferred_lft forever
3: enp0s6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:1c:42:73:84:98 brd ff:ff:ff:ff:ff:ff
    inet 192.168.193.128/16 brd 192.168.255.255 scope global dynamic enp0s6
       valid_lft 85656sec preferred_lft 85656sec
    inet6 fe80::21c:42ff:fe73:8498/64 scope link
       valid_lft forever preferred_lft forever

So, enp0s5=192.168.228.95 and enp0s6=192.168.193.128.

If I install and start k3s with no options, I have k3s binded to both 192.168.228.95 and 192.168.193.128, which is normal.

If I try to attach only to one interface, enp0s6/192.168.193.128, the server always respond for http/https on both interfaces.

I've tried a lot of parameters, the last one for this issue is:

k3s server --node-ip=192.168.193.128 --flannel-iface=enp0s6 --bind-address=192.168.193.128

You can see on the following capture, both terms on the right has a 404 page coming from k3s ingresses. If only one interface was successfully bound, only the one on the bottom right should display this 404 message.

I've done this with k3s version v1.21.3+k3s1 (1d1f220f) using the default install script.

Thanks for your help.

[dfoxg]

Are here any updates? I also have trouble to bind the node-port to my local ip:
./k3s-install.sh --bind-address 10.0.0.5 --node-name l0005 --node-ip 10.0.0.5 --flannel-iface vpncloud0 --node-external-ip 10.0.0.5 --advertise-address 10.0.0.5 --kube-proxy-arg bind-address=10.0.0.5

 netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 127.0.0.1:10248         0.0.0.0:*               LISTEN      290124/k3s server
tcp        0      0 127.0.0.1:10249         0.0.0.0:*               LISTEN      290124/k3s server
tcp        0      0 127.0.0.1:10251         0.0.0.0:*               LISTEN      290124/k3s server
tcp        0      0 10.0.0.5:6443           0.0.0.0:*               LISTEN      290124/k3s server
tcp        0      0 127.0.0.1:10252         0.0.0.0:*               LISTEN      290124/k3s server
tcp        0      0 127.0.0.1:6444          0.0.0.0:*               LISTEN      290124/k3s server
tcp        0      0 0.0.0.0:31888           0.0.0.0:*               LISTEN      290124/k3s server
tcp        0      0 127.0.0.1:10256         0.0.0.0:*               LISTEN      290124/k3s server
tcp        0      0 127.0.0.1:10258         0.0.0.0:*               LISTEN      290124/k3s server
tcp        0      0 127.0.0.1:10010         0.0.0.0:*               LISTEN      290161/containerd
tcp        0      0 0.0.0.0:31067           0.0.0.0:*               LISTEN      290124/k3s server
tcp6       0      0 :::10250                :::*                    LISTEN      290124/k3s server

[rbrtbnfgl]

The issue is not related to kube-proxy or the bind-address. We are using portmap cni by default to forward the traffic of the host to containers. Portmap should be configurable using the cni conf file and by default is forwarding all the traffic.
The portmap configuration can be modified using a custom cni-config with the flag --flannel-cni-conf with the path of the custom file. From the portmap doc https://www.cni.dev/plugins/current/meta/portmap/ you should add the conditionsV4 option. A sample file that should disable the forwarding from the other interface could be:

{
  "name":"cbr0",
  "cniVersion":"1.0.0",
  "plugins":[
    {
      "type":"flannel",
      "delegate":{
        "hairpinMode":true,
        "forceAddress":true,
        "isDefaultGateway":true
      }
    },
    {
      "type":"portmap",
      "capabilities":{
        "portMappings":true
      },
      "conditionsV4": ["!", "-d", "192.168.228.95"]
    },
    {
      "type":"bandwidth",
      "capabilities":{
        "bandwidth":true
      }
    }
  ]
}

I used @davinkevin configuration as example. This file should not forward the traffic that has 192.168.228.95 as destination.

[caroline-suse-rancher]

I'm going to convert this to a discussion since it seems to be a specific problem people are having that's unrelated to a k3s bug

[m00n]

Did anyone get the example flannel config to work? I've tried, but port 80 and 443 still get redirected to k3s for all IPs configured for my network interface. I'm a bit stuck here, because iptables -L is over 700 lines long with all Docker and Kubernetes rules and I have a hard time figuring out how to free up those two ports.