Home

Overview

Scalable function is a lightweight function that has a parent PCI function on which it is deployed. mlx5 scalable function has its own function capabilities and its own resources. This means a scalable function has its own dedicated queues(txq, rxq, cq, eq). These queues are neither shared nor stolen from the parent PCI function.

There is no special support needed from system BIOS to use scalable functions. Scalable functions co-exist with PCI SR-IOV virtual functions. Scalable functions do not require enabling PCI SR-IOV.

mlx5 scalable function is implemented using Linux kernel subfunction.

Refer to step by step guide on how to use them with upstream Linux kernel.

Internals

• When scalable function is RDMA capable, it has its own QP1, GID table and rdma resources neither shared nor stolen from the parent PCI function.

• A scalable function has dedicated window in PCI BAR space that is not shared with the other scalable functions or parent PCI function. This ensures that all class devices of the scalable function accesses only assigned PCI BAR space.

• A scalable function supports eswitch representation through which it supports tc offloads. User must configure eswitch to send/receive packets from/to scalable function port.

• Scalable functions share PCI level resources such as PCI MSI-X IRQs with their other scalable function and/or with its parent PCI function.

For more information about subfunction refer to the kernel's subfunction documentation and mlx5 subfunction documentation.

Overall design approach discussed with wider community, developers, vendors at netdev conference 0x14. Its presentation contains slides, discussion and Q&A.

SFs vs VFs

1. SFs are deployed in unit of one unlike SR-IOV VFs which are enabled all together. When a new container is spawned, at that point needed SF can be created and deployed.

2. SFs do not have to implement full PCI config space, reset, registers. This makes the device light weight.

3. SFs share MSI-X vectors with owner PCI PF and other peer SFs. This reduces the demand on total number of vectors in hardware and platform interrupt controller.

4. SFs can be deployed from the DPU in cloud native manner. This avoid cross system synchronization in representor setup. In other words, administrator can fully setup the SF, its representor, QoS, needed encap rules, needed parameters of SF, before hotplugging the SF in the host environment where it will be used.