Make SRIOV VFs to push/pop tags like vlan or vxlan

[denis-tingaikin]

We need to rework SRIOV VFs looking at experience with vlan and vxlan push/pop tags mechanic. See at https://github.com/networkservicemesh/sdk-kernel/pulls?q=is%3Apr+is%3Aclosed+vlan+

[edwarnicke]

Be careful. What we are doing here is having the forwarder program the PF (physical interface) so that the VF (virtual interface) push/pops the correct vlan or vxlan tags. This should be true whether or not the mechanism the VF is using is kernel or vfio. Also, in contrast to other efforts... the vlan tag is not to be pushed by software in the kernel, we are programming the PF so that the particular VF is having its vlan tag push/popped in hardware by the NIC.

[glazychev-art]

@edwarnicke
Do I understand correctly that we need to implement a feature that allows to send VLAN-tagged traffic by virtual functions (VF)?

We currently only have one VLAN on the Packet cluster, which we configure when the server starts. And the server's network configuration does not need to be VLAN-aware (see https://deploy.equinix.com/developers/docs/metal/layer2-networking/layer2-mode/#server-configuration-unbonded).

For this feature, we need to explicitly tag the traffic. Am I right?

[glazychev-art]

Current state:

1. Solved the problem with the VLAN on the packet. (for unknown reasons, at least 2 VLANs must be used)
2. Manually configured SriovKernel2Noop example with tagged VF.
3. [In progress] Manual configuring Vfio2Noop

Unresolved issues:

1. Should we consider adding a new application? For example, remotevlan has cmd-nse-remote-vlan that configures gw and vlan. But our current SriovKernel2Noop doesn't have something similar, we use simple cmd-nse-icmp-responder.
2. Should we reuse the current chain elements for configuration? For example, ethernetcontext is very similar what we need.
3. Calico CNI plugin (not VPP) on kubernetes cluster may stop working after manipulating vlan/ip addresses. Need to test more.
4. Because we will now assign VLANs in deployment-k8s (in the endpoint configuration), there is a problem of VLANs overlapping when running a default and Calico-VPP clusters at the same time. These tests from different clusters will affect each other. Edited: I think we can consider using different facility packet option for the default and Calico-VPP CNI

[glazychev-art]

Current state:

1. Run Vfio2Noop example with vlan tagged VFs.

Question:
When we use VLAN tags, we set them in the NSE config. For example:

• Remote VLAN - https://github.com/networkservicemesh/deployments-k8s/blob/main/examples/remotevlan/patch-nse.yaml

metadata:
  name: nse-remote-vlan
  ...
            - name: NSM_SERVICES
              value: "finance-bridge { vlan: 100; via: gw1}"

• Vfio2Noop will look like:

metadata:
  name: nse-vfio
  ...
            - name: NSM_SERVICE_NAMES
              value: "vfio2noop@worker.domain: { addr: 0a:55:44:33:22:11; vlan: 1044 }"

They use cmd-nse-remote-vlan and cmd-nse-vfio which parse the config.

The point is that for SriovKernel2Noop we need something similar because it uses a simple cmd-nse-icmp-responder. Should we add a new application, or modify existing ones? Perhaps we need to consider cmd-nse-vlan-vpp?

@denis-tingaikin @edwarnicke
Any thoughts would be helpful!

[glazychev-art]

What else needs to be done on this issue:

1. Merge prepared PRs (see above)
2. Create a new or modify existing NSE app (see Make SRIOV VFs to push/pop tags like vlan or vxlan #489 (comment))
3. Modify our deployment-k8s example (most of it is already done locally)
4. Modify integration-k8s-packet setup (most of it is already done locally)
5. Testing

[glazychev-art]

Vlan tagged Vfio2Noop example

Passing a Request in the Vfio2Noop test with VLAN tag:

Forwarder-1 uses netlink to add a VLAN-tag (see ethernetcontext)

@edwarnicke
Is this diagram correct for Vfio2Noop?

[glazychev-art]

Looks like done - networkservicemesh/deployments-k8s#9129