Allow n-hop routing

[rakshasa]

Currently we only support 1-hop routing due to the limitations of the current route link implementation.

Expand this to allow for 2-, and n-hop routing between networks.

[rakshasa]

• We know if we need to hop more if the dest ip is outside of the subnet of the egress interface's network ranges.
• Since we do not have routing tables, create the gateway interface-to-interface step based on routes on the intermediate network.
• The interface-to-interface portion needs to include translation and filtering.
• We can have a catch-flow for egress (ingress?) interfaces/route_links at each step so that they can be dynamically loaded.

[rakshasa]

Flows for n-hop routing

Network layout

nw-vnet2 <-(10.103.0.1)-rl2-(10.102.0.5)-> nw-vnet1 <-(10.102.0.1)-rl1-(192.168.71.foo)-> nw-public2

Finalized Hop Flows

(37): TABLE_ROUTE_EGRESS_INTERFACE_1
 37-00 => SWITCH(0x0)               actions=drop
 37-10 => INTERFACE(0x8)[0x12]     metadata=TYPE_INTERFACE(0x9)                           actions=set_field:IF(x8)_mac->eth_src,write_metadata:TYPE_NETWORK(0x4),goto_table:TABLE_TABLE_ROUTE_HOP_NETWORK_1(40)
 37-10 => INTERFACE(0x9)[0x12]     metadata=TYPE_INTERFACE(0x9)                           actions=set_field:IF(x9)_mac->eth_src,write_metadata:TYPE_NETWORK(0x4),goto_table:TABLE_TABLE_ROUTE_HOP_NETWORK_1(40)
 37-10 => INTERFACE(0xf)[0x12]     metadata=TYPE_INTERFACE(0xf)                           actions=set_field:IF(xf)_mac->eth_src,write_metadata:TYPE_NETWORK(0x3),goto_table:TABLE_TABLE_ROUTE_HOP_NETWORK_1(40)
 37-44 => INTERFACE(0x8)[0x12]     ip,metadata=TYPE_INTERFACE(0x8),nw_dst=10.102.0.0/24   actions=set_field:IF(x8)_mac->eth_src,write_metadata:TYPE_NETWORK(0x4),goto_table:TABLE_ARP_TABLE(40)
 37-44 => INTERFACE(0x9)[0x12]     ip,metadata=TYPE_INTERFACE(0x9),nw_dst=10.102.0.0/24   actions=set_field:IF(x9)_mac->eth_src,write_metadata:TYPE_NETWORK(0x4),goto_table:TABLE_ARP_TABLE(40)
 37-44 => INTERFACE(0xf)[0x12]     ip,metadata=TYPE_INTERFACE(0xf),nw_dst=192.168.60.0/24 actions=set_field:IF(xf)_mac->eth_src,write_metadata:TYPE_NETWORK(0x3),goto_table:TABLE_ARP_TABLE(40)

(38): TABLE_ROUTE_HOP_NETWORK_1
 38-20 => INTERFACE(0x8)[0x12]     ip,metadata=TYPE_NETWORK(0x4),nw_dst=0.0.0.0/0       actions=write_metadata:INTERFACE(0x8),set_field:IF(x8)_mac->eth_dst,goto_table:TABLE_TABLE_ROUTE_INGRESS_TRANSLATION_2(41)
 37-22 => INTERFACE(0xf)[0x12]     ip,metadata=TYPE_NETWORK(0x3),nw_dst=0.0.0.0/0       actions=write_metadata:INTERFACE(0x11),goto_table:TABLE_ARP_TABLE_GATEWAYS(xx)
 38-36 => INTERFACE(0x9)[0x12]     ip,metadata=TYPE_NETWORK(0x4),nw_dst=10.103.0.0/16   actions=write_metadata:INTERFACE(0x9),set_field:IF(x9)_mac->eth_dst,goto_table:TABLE_TABLE_ROUTE_INGRESS_TRANSLATION_2(41)
 38-44 => INTERFACE(0x8)[0x12]     ip,metadata=TYPE_NETWORK(0x4),nw_dst=192.168.60.0/24 actions=write_metadata:INTERFACE(0x8),set_field:IF(x8)_mac->eth_dst,goto_table:TABLE_TABLE_ROUTE_INGRESS_TRANSLATION_2(41)

(41): TABLE_ROUTE_INGRESS_TRANSLATION_2
 unchanged

NETWORK(0x3) is physical with external gateway
NETWORK(0x4) is virtual

Verify that reflection remains set.

For non-vnet gateway on a network, an external interface must be created and its routes added.

Dropping eth_src/dst in T_R_EGRESS_INTERFACE and T_R_HOP_NETWORK

When a packet is passed to T_R_HOP_NETWORK, it is known to only pass through L3 routing. The original L2 addresses will not leak when it exits the last gateway, nor can any L2 address translations happen. This means that the mac addresses of the src/dst gateways during hops do not matter.

Route priority unique db entries

Note that the priorities of flows depend on their subnet prefix value, which is '20 + prefix'.

We might have overlap if multiple interfaces on the same network share the exact same route subnet+prefix. Thus we need routing priorities, such that the flow priority will be '20 + prefix + (priority * 40)'.

Routes db table include the network column, so we would add a priority column. The [network,subnet,prefix] columns would be unique.

T_R_HOP_NETWORK Handling External Routes

In the case of network with routes for virtual gateways adding hops is simple. This is not the case for network with non-vnet gateways.

When the next gateway on a network is an physical/unmanaged interface, we currently rely on a simple config option in 'vna.conf' and hacked up arp lookup. This means we need to introduce a route type that applies to external physical interfaces, where eth_src/dst is properly set and the packet is passed to network dst handling.

Note: The packet will already be egressing from a pre-determined datapath as gateway interfaces on physical networks are tied to one and only one datapath.

Avoiding route reflection

In TABLE_ROUTE_HOP_NETWORK there may be cases where a packet is routed back down the route it came from.

To avoid this we should use value_pair metadata with 'network_id,previous_route_id' and have a dropflow if both are matched. These dropflows would be one priority higher than the regular route flow.

While this does not guarantee the flowtable to be loop-free for loops with 2-steps and above, the limit on hops ensures these edge-cases do not cause harm. It might even be reasonable to skip the above safe-guard for 1-step loops.