[Feature Request] Support IPv6 nexthops for IPv4 routes

[SkalaNetworks]

Description

Currently, putting an IPv6 gateway for an IPv4 route yields the follow error:
failed to add route '{0.0.0.0 00000000} via fd00:0:641:1::1 dev net1': gateway, source, and destination ip are not the same IP family

Route annotation:
'[{"dst":"::/0","gw":"fd00:0:641:1::1"},{"dst":"0.0.0.0/0","gw":"fd00:0:641:1::1"}]'

This would be useful for NAT gateways where BGP extended-nexthop is active. This issue is associated to PR #4704 where extended-nexthop is proposed.

Who will benefit from this feature?

DCs with IPv6 only fabrics would benefit from this feature.

Anything else?

No response

[oilbeater]

The cni will set the route in pod netns. I am not sure if this is a valid format for linux route.

[SkalaNetworks]

From a Linux standpoint, it is perfectly valid:
ip route add 1.1.1.1 via inet6 fd00::1

This is useful when the underlay is an IPv6 fabric, something more and more common especially in the datacenter.

[bobz965]

maybe it is a new feature. In most case, subnet should use it gw inside its cidr.

in this case , the gw is outside subnet cidr.

[SkalaNetworks]

There's no particular reason for the GW to be inside the same subnet, at its heart, this is just L2 forwarding of a packet, and indicating an L3 address is purely done to look up the ARP table and determine who should be the recipient. This is also why the BGP capability for extended-nexthops was introduced and is now used in IPv6-only datacenter fabrics.

My PR for extended-nexthop has been merged in master, this is more of a proposition to further help Kube-OVN operate in IPv6-only envs.

[oilbeater]

failed to add route '{0.0.0.0 00000000} via fd00:0:641:1::1 dev net1': gateway, source, and destination ip are not the same IP family

@SkalaNetworks this error log is generated by netlink library. If netlink not support this, I think we cannot easily set this route.

Also when I try ip route add 1.1.1.1 via inet6 fd00::1 in ubuntu 22.04.1, it also generate error message RTNETLINK answers: No route to host

[SkalaNetworks]

You need a route to the default gateway, c.f https://forum.vyos.io/t/ability-to-have-ipv6-nexthops-for-ipv4-static-routes/13601

Note: This is a VyOS post discussing linux ipv6 nexthop capabilities

[oilbeater]

@SkalaNetworks Thanks for the information. I finally understand what IPv6-only datacenter fabrics is. I think add a netlink route family option can resolve this.

[zhangzujian]

upstream issue: vishvananda/netlink#908

[oilbeater]

There is a bug exist in netlink, which wrongly use src family as the dst family to validate gateway.