Working Calyx Implementation of AXI Read channels

[nathanielnrn]

This PR adds a working implementation of the read portion of an AXI interface according to the latest spec and RTL kernel specifications for XRT.

Currently the tests are a bit of a pain to run, requiring some replacement of the verilog code generated. To this end here is a zipped .fst showing the waveforms of the three tests, in case anyone is interested.

The calyx is heavily commented, so hopefully not too bad to go through. However I know this is a lot so happy to hop on a call sometime to go over things.

Some thoughts about this so far:

1. There are definitely some inefficiencies in our interface. The immediate one that comes
   to mind is that handshakes require 2 cycles instead of one. Servicing reads (writing them to the seq_mem_d1 in the calyx program takes ~9 cycles. And we also utilize a data_bus only as wide as our memory (32 bits for an eventual integer dot-product test case).
2. Some of the grouping we were initially trying to have, a blocking group, a servicing group, an assert and deassert group, had to be combined together due to the time it takes to transfer between groups. @sampsyo and I spoke very broadly about there possibly being future work here to enable faster transitions between groups (1 cycle as opposed to 2 right now IIRC).

These will be added to #1733 and tracked there but for now some TODOs:

1. Make cocotb tests interface with runt
2. Create tests that perform more than 1 transaction
3. Work on write channels.

Hopefully this last one will be a bit more straightforward now that there is some infrastructure in place and working channels to pattern match off of.

cc @andrewb1999 @rachitnigam @sampsyo

[sampsyo]

Awesome!!!!! Really cool that this is minimally working for the read side.

To slightly expand on this:

Some of the grouping we were initially trying to have, a blocking group, a servicing group, an assert and deassert group, had to be combined together due to the time it takes to transfer between groups.

I actually think this is a super interesting problem that has implications for static/dynamic Calyx assignments. Artistically, we would like to be able to manage an AXI transaction with a control program like this:

while <more transactions to do> {
  seq {
    assert_ready;  // tell the remote side we are ready for their data
    wait_for_valid;  // wait for them to send us stuff
    par {
      do_transfer;  // actually do things the data on the bus
      deassert_ready;  // tell them to wait while we process the data
    }
  }
}

The problem is that AXI, of course, is super cycle-timing-sensitive. Remember that one of the core invariants is:

On any cycle when both ready=1 and valid=1, a transaction occurs.

So the remote side can schedule these transaction cycles back-to-back, i.e., 3 transactions can happen in exactly 3 cycles. Most saliently for us, say the remote side just keeps its valid signal high all the time and relies on us to assert/deassert ready to do transfers. Then, if we need to take some time to process a transfer, we have to be absolutely sure we only assert ready for one cycle at a time.

The problem with dynamic Calyx is that it offers no guarantees about how long it takes to go from wait_for_valid above to deassert_ready. And in practice, it actually does waste at least one cycle, meaning that the remote side sees us do two transfers when we only wanted one.

The big upshot of static Calyx is that it does offer these guarantees, so we would like to be able to just sprinkle the static everywhere get the timing we want. There is a critical problem, however: the wait_for_valid group is fundamentally dynamic. Our "dynamic islands" approach does not support sequencing a dynamic group with a static one with a zero-cycle transition latency.

So the big design question that I think this opens up is: is there some rule we can make up that would allow this kind of sequencing? That is, to allow some version of seq { a ; b } where a is dynamic and b is static where we can guarantee that b runs in the same cycle where a says it is done. It seems hard to do in general but maybe there is something we can do for restricted cases?

[sampsyo]

This all looks great. For others reading this, the current thing is a hand-written, standalone Calyx program; getting this tested and working is a stepping stone toward "wiring it up" to core Calyx logic.

What do you think about renaming the top-level directory from fpgas to yxi to reflect the broader scope of work here?

[rachitnigam]

@sampsyo Interesting point! Do you want to move this into an issue? Seems like there is some level of deeper semantics tweaking needed to make this work because we'd need to think about providing timing guarantees in a dynamic context.

[sampsyo]

FYI, I think this is good to merge whenever you're ready, @nathanielnrn!