Option to use veth instead of dummy for DPDK support

[krsna1729]

I did not follow up on your comment here k8snetworkplumbingwg/sriov-cni#37 (comment) but one of the reasons I used veth here was some slowpath processing can happen in the kernel. We used veth to inject on one side from DPDK app. The kernel would receive them on the other end of veth and then craft any replies or update its state.

From what I understand dummy interface is not capable of this. What are you thoughts on providing a veth option?

[Levovar]

if you want slowpath synch why don't you simply ask an additional slowpath interface for this, i.e. IPVLAN for example?
the dummy interface is purely for conveying CaaS allocated network resources to the application but is not intended for communicating on top of it. from my perspective a slowpath interface is an extra, functional network interface you need, and it shouldn't come automatically with a VFIO bound VF.
it is something your app needs in addition to the VF, so IMHO it is better to explicitly express this need in your manifest

[krsna1729]

The use case is to direct arp, icmp, dynamic route protocol packets targeting the fastpath IP of the app etc., to the kernel i.e., slowpath and handle transit traffic routing in fastpath. If this can be done using an extra ipvlan, i am interested in knowing that.

Basically we want to offload as much control packet processing targeted to fastpath's IP to the kernel, instead of re-implementing it within the app/fastpath

[Levovar]

yeah I'm familiar with terminology, we use the same internally as well
I don't see why an IPVLAN or MACVLAN interface wouldn't work for your case, depends whether you need L2 or L3 switching
you just need some intelligence/configurability in the app to be able to tell it which kernel IF should be used for sycnhing, but I'm assuming something like that is anyway needed
and you could identify the kernel synch interface based on its name, which is configurable in DANM from the network's Spec.Options.container_prefix

but please let me know if you find a problem with this approach!

[krsna1729]

Is there an example of such a pod?

Who is splitting the traffic and based on what is it decided that arp, icmp go to macvlan vs GTPU traffic goes to SRIOV interface?

Currently we have a filter in our app we do this splitting. SRIOV -> host-traffic filter -> veth. This veth is dpdk end. The kernel end mirrors the VF mac and has IP and route details.

https://github.com/omec-project/upf-epc/blob/master/conf/spgwu.bess#L148-L170

[Levovar]

I'm sure there is, but the only one I know of is unfortunetaly proprietary :)

so just for my reference, how do exactly the veth pair look like in your example? I understand that both are created inside the container netns, but I'm not sure how would that work exactly
veths are usually connected to a bridge, but I maybe not in this case? or the "kernel end" still is, otherwise how would connectivity work?
also what's the IP/MAC of the DPDK end? nothing?

[krsna1729]

They are not connected to bridge in this case. The DPDK end there is no IP and MAC is dont care.

[Levovar]

BTW how does the application know which veth interface it should use? or simply "the one not having an IP is mine"?

[krsna1729]

The app simply looks up the peer of the CNI configured veth end. Since we know the "CNI/linux" end's name we can do this -

detect_mode
peer_by_interface