创建的underlay ip的容器无法访问网络

[i17c]

Bug Report

Type: bug report

What happened

创建的underlay ip的容器无法访问网络
host主机的ip：
100.107.167.16/24
vlan：900
gateway：100.107.167.55

容器的ip：
100.107.169.155/26
vlan: 900
gateway: 100.107.169.183

What you expected to happen

正常情况下，希望启动的容器可以访问网络。

How to reproduce it (as minimally and precisely as possible)

在特定主机启动一个同一个vlanID，但不同网段，不同子网掩码，不同gateway的underlay ip的容器。

Anything else we need to know?

设定的vlanID=0

主机上网卡信息：
6: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 98:03:9b:0f:0a:ba brd ff:ff:ff:ff:ff:ff
inet 100.107.169.155/26 brd 100.107.169.191 scope link noprefixroute bond0
valid_lft forever preferred_lft forever
inet 100.107.167.16/26 brd 100.107.167.63 scope global bond0
valid_lft forever preferred_lft forever

5074: bond0.vxlan4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc prio state UNKNOWN group default qlen 1000
link/ether 98:03:9b:0f:0a:ba brd ff:ff:ff:ff:ff:ff
inet 100.107.167.16/26 brd 100.107.167.63 scope global noprefixroute bond0.vxlan4
valid_lft forever preferred_lft forever
inet 100.107.169.155/26 brd 100.107.169.191 scope global noprefixroute bond0.vxlan4
valid_lft forever preferred_lft forever

路由信息：

default via 100.107.167.55 dev bond0
10.0.0.0/8 via 100.107.167.55 dev bond0
11.0.0.0/8 via 100.107.167.55 dev bond0
25.0.0.0/8 via 100.107.167.55 dev bond0
26.0.0.0/8 via 100.107.167.55 dev bond0
30.0.0.0/8 via 100.107.167.55 dev bond0
100.64.0.0/10 via 100.107.167.55 dev bond0
100.107.167.0/26 dev bond0 proto kernel scope link src 100.107.167.16
169.254.0.0/16 dev bond0 scope link metric 1006
172.16.0.0/12 via 100.107.167.55 dev bond0
192.168.0.0/16 via 100.107.167.55 dev bond0

ip route show table 39999:

10.244.0.32 dev hybr95563dc8652
10.244.0.54 dev hybrf8ec67d57f7
10.244.0.57 dev hybr9be5d0bbdbb
10.244.0.192 dev hybrd1f58fd0298
10.244.1.37 dev hybr0255ebd2f43
100.107.169.155 dev hybrdb85980a200

ip rule:

0: from all lookup local
1: from all lookup 39999
2: from all lookup 40000
3: from all fwmark 0x20/0x20 lookup 40001
4: from 10.244.0.0/16 fwmark 0x0/0x4040 lookup 10000
5: from 100.107.169.128/26 fwmark 0x0/0x4040 lookup 10001
32766: from all lookup main
32767: from all lookup default

主机上 arp 规则：
"arp -a|grep 100.107.169.155"
? (100.107.169.155) at 00:00:5e:00:01:01 [ether] on bond0
? (100.107.169.155) at <from_interface> PERM PUB on bond0.vxlan4
? (100.107.169.155) at <from_interface> PERM PUB on bond0

同正常节点（underlay ip vlanID不同的情况下正常）缺少一条规则，如下：
? (26.61.248.23) at 02:12:34:23:de:d2 [ether] on hybr0da5c475b21

pod的ip为： 100.107.169.155
启动正常，但是无法访问网络。
容器内部情况：

#arp
Address HWtype HWaddress Flags Mask Iface
gateway (incomplete) eth0

其他正常节点（underlay ip vlanID不同的情况下正常）：

#arp
Address HWtype HWaddress Flags Mask Iface
gateway ether ee:ee:ee:ee:ee:ee C eth0
26.55.138.196 ether ee:ee:ee:ee:ee:ee C eth0

Environment

• hybridnet version: v0.8.8

• OS (e.g. cat /etc/os-release): Alibaba Group Enterprise Linux Server 7.2 (Paladin)

• Kernel (e.g. uname -a): Linux z97c15213.cloud.ea134 4.19.91-013.ali4000.alios7.x86_64 init code #1 SMP Thu Jan 6 15:04:24 CST 2022 x86_64 x86_64 x86_64 GNU/Linux

• Kubernetes version: 1.28.2

• Install tools: helm

• Others:
  host主机的ip：
  100.107.167.16/24
  vlan：900
  gateway：100.107.167.55

    容器的ip：
    100.107.169.155/26
    vlan: 900
    gateway: 100.107.169.183
  

[mars1024]

Hi, we have found the root cause, vtep link is generated by overlay network, during creation, the address of vtep link will borrow from default link, but unfortunately, it borrow a wrong address (the enhanced address which is generated by underlay network), so if we want to fix this, the enhanced address should be ignored.

We will fix this in the next minor version.