URGENT: Existing support for linuxptp on open-nic driver

[Sandman0196]

In our project, we already have synchronized the system clock and the shared and SmartNIC clocks in VMs using kvm_ptp on a Virtual Machine as shown below. It utilizes the ptp hardware support on the NICs, both in the actual hardware and the VMs to synchronize the system and the NIC clocks with ptp. It does that through ptp_daemon and the pcie buses (/dev/ptpX, dev/ptpY, etc). It is possible to do so with hardware devices which already have hardware support for ptp synchronization.

Now, we are using Alveo U280 accelerator cards for running P4 program to control the data plane. In that, we are trying to get hardware timestamps from the accelerator cards which would be ptp synchronized as mentioned above. But the issue is, U280 doesn’t have ARM core as the MPSoC so that the software driver used to control ptp (https://github.com/Xilinx/linux-xlnx/blob/master/drivers/ptp/Kconfig) cannot be directly used. We don’t know how to control the ptp from the OS. In MPSoC, Xilinx developed a specific linux-ptp driver that can be deployed on the MPSoC’s Arm core to control the ptp service on the hardware block (CMAC). I don’t think that we can directly use this specific driver to control the ptp on the CMAC.

So, do you have an existing support for linuxptp on open-nic driver? If not, is it possible to do that with the open-nicshell or cmac driver? Do we have any support for that on the driver? Can we control it from a Xilinx ptp library, or a memory file or pcie bus? How can we control the ptp services on the CMAC hardware IP from our x86 based linux host through the PCIE bus?

Any sort of help or pointers would be really helpful. Thanks in advance.

[cneely-amd]

Hi @Sandman0196,

The short answer is that OpenNIC shell doesn't make any attempt to use PTP.

However, I saw there is mention of PTP in this CMAC-related user guide, a very short section discussing timestamps: https://docs.xilinx.com/r/en-US/pg165-cmac/Transmit-1588-Insertion-and-Timestamp-Function.

For open-nic-driver there are no functions in the current driver related to 1588.

However, I understand from your message is you found on the web an MPSoC software driver for the ARM cores that exists, link that you gave.

I wonder if it might be possible to run that sort of driver using a soft processor core like a Microblaze, but that is just a quick thought without knowing what details might be involved.

I asked someone knowledgeable, and they commented, "There are targeted reference designs for 10G/25GE soft (ZCU102, ZCU670) and MRMAC (10G/25GE) (VPK190) with driver system support found on the web. AMD has a partnership with another vendor to enable larger solutions."

Hope this helps.

Thanks,
--Chris

[lmunch]

Hi @Sandman0196

The QDMA descriptor can be customized, so I would assume you could fit a timestamp into those and extend OpenNIC accordingly. I would also expect the PHC(ptp) driver and ToD IP from the MPSoC reference design could be a good starting point. I have not tried the above (although it sounds like fun), but that would be how I would attack the task.

If you want to go the MPSoC route, then this SmartNIC has OpenNIC on the host and a very accurate PTP implementation running on the ARM. The ToD on the ARM is then exposed (readonly) to the host though a modified OpenNIC driver:
https://www.silicom.dk/product/timesync-smartnic-fpga-fb4xxvgz21d/

Regards
Lars