[fud] Revamp Xilinx fpga stage to work for both emulation and execution

[sampsyo]

As it stands, there are two fud stages available to run Xilinx xclbin-compiled bitstreams:

• In emulation.py, there is a stage that uses a host C++ source file to run the whole shebang. This requires you to write the C++ testbench yourself and provide it as a configuration option. This is what I have documented so far in the xilinx.md chapter (only because it was the first one I've tried).
• In execution.py, there's a very different approach in the fpga stage: it imports PyOpenCL and uses that to load up and invoke the kernel directly. So there is no host C++ program; the fud Python process itself talks to the device.

As the names imply, I think @sgpthomas originally meant for these two stages to cover emulation & actual FPGA execution, respectively. But I have ben able to get the fpga stage to do emulation (in short, XRT respects the environment variable setting XCL_EMULATION_MODE=hw_emu and does emulation).

So my proposal here is to remove the emulation.py route and replace it with a spruced-up version of the fpga stage. Specifically, the plan would be this:

• We would create a separate, standalone (not part of fud) tool for running xclbin files. The fud stage would then run this tool as a subprocess. This tool would take care of loading up the data file, loading the xclbin, invoking it, and collecting the results. Although it seems odd, there are several reasons to propose a standalone tool:
  • It would make drop-in replacement easy. We could prototype the tool using PyOpenCL, for example, and eventually replace it with a C++ or Rust program that directly uses the XRT API.
  • It would easily allow remote execution, like the other Xilinx-related fud stages.
  • It would provide an easy manual fallback for running xclbin files without using fud.
  • Probably the most important thing: it would let fud do the dirty work of setting up the environment before invoking the tool (e.g., by sourcing the XRT setup script). This way, fud users would not need to bother with this nonsense themselves, as they currently do.
• An "easy" prototype could reuse the PyOpenCL code that currently exists in the fud stage. But then I think we should somewhat quickly move on to replacing OpenCL with the pyxrt bindings instead. This would give us better/more direct control over the emulation configuration, etc., because it surfaces more Xilinx-specific APIs than the standard OpenCL APIs.
  • For example, we could avoid writing an xrt.ini file for configuration and instead use xrt::ini::set directly.

[sampsyo]

Also, I think good follow-on work to this revamping would be configuring this tool to produce waveform traces during Xilinx emulation. Unfortunately, Xilinx has their own special waveform file format (WDB). We would need to figure out how to use these or how to convert them to VCD.

[nathanielnrn]

Stumbled upon this, think it can be closed? @sampsyo @rachitnigam

[rachitnigam]

Sounds good!