[fud] Make SSH optional in Xilinx emulation stage (#852)

* Start refactoring generic sandbox utility

LocalSandbox is a new alternative to RemoteExecution for running
commands locally. It's not a drop-in replacement (maybe that should come
in the future), but it offers morally equivalent functionality.

* Fix some configuration stuff

* Improve an error message

I don't know why "provide an input file" was suggested, so I took that
out. Also, properly interpolate the variable into the example flag.

* Document what we know so far

* Black formatting

* Simplify emulation command execution

And also make them work locally, with a utility.

* Avoid some hard-coding

* Document wdb stage configuration

* Fix for name change from #851

docs/fud/synthesis.md

@@ -83,6 +83,14 @@ The options for `mode` are `hw_emu` (simulation) and `hw` (on-FPGA execution).
 The device string above is for the [Alveo U50][u50] card, which we have at Cornell, but I honestly don't know how you're supposed to find the right string for a different FPGA target.
 Hopefully someone will figure this out and document it in the future.
 
+To use hardware emulation, you will also need to configure the `wdb` stage.
+It has similar `ssh_host`, `ssh_username`, and `remote` options to the `xclbin` stage.
+You will also need to configure the stage to point to your installations of [Vitis][] and [XRT][], like this:
+
+    [stages.wdb]
+    xilinx_location: /scratch/opt/Xilinx/Vitis/2020.2
+    xrt_location: /opt/xilinx/xrt
+
 ### Compile
 
 The first step in the Xilinx toolchain is to generate [an `xclbin` executable file][xclbin].
@@ -97,6 +105,20 @@ By default, the Xilinx tools run in a temporary directory that is deleted when `
 To instead keep the sandbox directory, use `-s xclbin.save_temps true`.
 You can then find the results in a directory named `fud-out-N` for some number `N`.
 
+### Emulate
+
+You can also execute compiled designs through Xilinx hardware emulation.
+Use the `wdb` state as your `fud` target:
+
+    fud e -vv foo.xclbin -s wdb.save_temps true -o out.wdb
+
+This stage produces a Vivado [waveform database (WDB) file][wdb]
+Through the magic of `fud`, you can also go all the way from a Calyx program to a `wdb` file in the same way.
+There is also a `wdb.save_temps` option, as with the `xclbin` stage.
+
+You also need to provide a host C++ program via the `wdb.host` parameter, but I don't know much about that yet, so documentation about that will have to wait.
+Similarly, I don't yet know what you're supposed to *do* with a WDB file; maybe we should figure out how to produce a VCD instead.
+
 ### How it Works
 
 The first step is to generate input files.
@@ -131,3 +153,4 @@ This step uses the `v++` tool, with a command line that looks like this:
 [xclbin]: https://xilinx.github.io/XRT/2021.2/html/formats.html#xclbin
 [gen_xo]: https://github.com/cucapra/calyx/blob/master/fud/bitstream/gen_xo.tcl
 [u50]: https://www.xilinx.com/products/boards-and-kits/alveo/u50.html
+[wdb]: https://support.xilinx.com/s/article/64000?language=en_US

fud/fud/config.py

@@ -79,6 +79,7 @@
             "mode": "hw_emu",
             "ssh_host": "",
             "ssh_username": "",
+            "remote": None,
             "host": None,
             "save_temps": None,
             "xilinx_location": "/scratch/opt/Xilinx/Vitis/2020.2",

fud/fud/errors.py

@@ -68,10 +68,9 @@ class MissingDynamicConfiguration(FudError):
 
     def __init__(self, variable):
         msg = (
-            "Provide an input file or "
-            + f"`{variable}' needs to be set. "
+            f"`{variable}' needs to be set. "
             + "Use the runtime configuration flag to provide a value: "
-            + "'-s {variable} <value>'."
+            + f"'-s {variable} <value>'."
         )
         super().__init__(msg)
 

fud/fud/stages/remote_context.py

@@ -1,7 +1,9 @@
 import logging as log
 from pathlib import Path
 from tempfile import NamedTemporaryFile
+import shutil
 
+from fud.utils import TmpDir, FreshDir
 from .. import errors
 from ..stages import Source, SourceType
 
@@ -150,9 +152,7 @@ def copy_back(
             remote_tmpdir: SourceType.String,
             local_tmpdir: SourceType.Directory,
         ):
-            """
-            Copy files generated on server back to local host.
-            """
+            """Copy files generated on server back to local host."""
             with self.SCPClient(client.get_transport()) as scp:
                 scp.get(
                     remote_tmpdir, local_path=f"{local_tmpdir.name}", recursive=True
@@ -183,3 +183,65 @@ def fetch_file(
         local_path = fetch_file(client, remote_tmpdir)
         self._close(client, remote_tmpdir, keep_tmpdir=keep_tmpdir)
         return local_path
+
+
+class LocalSandbox:
+    """A utility for running commands in a temporary directory.
+
+    This is meant as a local alternative to `RemoteExecution`. Like that
+    utility, this provides steps to create a temporary directory,
+    execute programs in that temporary directory, and then retrieve
+    files from it. However, all this happens locally instead of via SSH.
+    """
+
+    def __init__(self, stage, save_temps=False):
+        self.stage = stage
+        self.save_temps = save_temps
+
+    def create(self, input_files):
+        """Copy input files to a fresh temporary directory.
+
+        `input_files` is a dict with the same format as `open_and_send`:
+        it maps local Source paths to destination strings.
+
+        Return a path to the newly-created temporary directory.
+        """
+
+        @self.stage.step()
+        def copy_file(
+            tmpdir: SourceType.String,
+            src_path: SourceType.Path,
+            dest_path: SourceType.String,
+        ):
+            """Copy an input file."""
+            shutil.copyfile(src_path, Path(tmpdir) / dest_path)
+
+        tmpdir = Source(
+            FreshDir() if self.save_temps else TmpDir(),
+            SourceType.Directory,
+        )
+        for src_path, dest_path in input_files.items():
+            if not isinstance(src_path, Source):
+                src_path = Source(src_path, SourceType.Path)
+            if not isinstance(dest_path, Source):
+                dest_path = Source(dest_path, SourceType.String)
+            copy_file(tmpdir, src_path, dest_path)
+
+        self.tmpdir = tmpdir
+        return tmpdir
+
+    def get_file(self, name):
+        """Retrieve a file from the sandbox directory."""
+
+        @self.stage.step()
+        def read_file(
+            tmpdir: SourceType.Directory,
+            name: SourceType.String,
+        ) -> SourceType.Path:
+            """Read an output file."""
+            return Path(tmpdir.name) / name
+
+        return read_file(
+            self.tmpdir,
+            Source(name, SourceType.String),
+        )