Create IDL Backend #1729

nathanielnrn · 2023-09-25T03:12:33Z

This PR creates a basic backend which takes in a calyx program and outputs information about the program's top level memories.
This is in the name of working on #1105 and #1603.

Specifically, for calyx programs like this

component main() -> () {
  cells {
    @external(1) A0 = std_mem_d1(32,8,4);                                                                                                                                                       
    @external(1) B0 = std_mem_d1(32,8,4);
  }
  ...
}

This backend produces

{
  "toplevel_name": "main",
  "memories": [
    {
      "name": "A0",
      "width": 32,
      "size": 8
    },
    {
      "name": "B0",
      "width": 32,
      "size": 8
    } 
  ]
}

A few things to note that probably warrant further discussion:

Currently this only reports on mem cells marked with @external. That may be all we need for interfacing with our programs via AXI?
Related to above, need to better understand our *memh goals and what *memh even does to understand what information we will need.
For multi-dimensional memories, this backend currently squashes the dimensions and outputs the product of each dimension's size as size. Would it be useful to retain that information instead?
Naming and location: Should probably rename this. Maybe CIDL? Also, this doesn't belong in xilinx/ but I'm hoping to get this out tonight for some early feedback and I still struggle with rust module/hierarchy issues, so will address later.

EDIT
Some answers to the above after a synch discussion:

Seems okay to march forward with only @external memories. Furthermore, only examining the toplevel memories that are external is okay, as marking memories that are parts of subcomponents in a calyx program is a bit messy and probably shouldn't be valid in the first place.
Basically, we want default calyx to only produce things that are synthesizable, which *memh is not. Our wrapper will wire up these *memh calls as needed to enable simulation.
This seems fine for now, as this corresponds with verilog representation of memories.
YXI is our name of choice. Will have .yxi file extensions and should be moved up a directory.

Some open questions: Naming and placement. This is probably not xilinx specific so should be moved up a level into just src. Also, naming is generic and not good and should be changed TODO: Only works withs d1 memories, but should not be hard to alter

Previously could only handle 1 dimension mems. NOTE: This flattens dimensions into a single size variable that is spit out. However it may be useful to maintain info about shape of memory? Need to think about this more

sampsyo

Looking good overall! I think my main substantive comment in here is that there are a few pre-existing utility functions that would probably benefit from being moved to their own file so they can be shared between the different backends that use them (to avoid copying-and-pasting the code for the utility functions).

sampsyo · 2023-09-29T20:38:14Z

calyx-backend/Cargo.toml

@@ -18,6 +18,7 @@ string-interner.workspace = true
 itertools.workspace = true
 linked-hash-map.workspace = true
 serde = { workspace = true }
+serde_json.workspace = true


I am ignorant enough of Rust workspace setups to not exactly know the difference between using serde_json.workspace = true here vs. just doing this right here:

serde_json = "1.0"

…and omitting this from the workspace-global Cargo.toml. Presumably there is a difference but I'm not sure what it is! Maybe this is the right thing; just thought I'd double-check this was the intended strategy (vs. what we do for csv and vast below).

Maybe @rachitnigam has an opinion on this? Tbh I've just been pattern matching from stuff I've seen around, the details of these workspaces elude me.

It probably doesn't matter much! I guess the upside of what you have now is that anyone else in the tree who ends up needing serde_json will use the same version.

calyx-backend/src/backend_opt.rs

calyx-backend/src/xilinx/idl.rs

sampsyo · 2023-09-29T20:49:10Z

calyx-backend/src/xilinx/idl.rs

+}
+
+// Returns Vec<String> of memory names
+fn external_memories(comp: &ir::Component) -> Vec<String> {


It looks like external_memories is identical to the definition in top-level.rs. (And get_mem_info is similar but does the "size flattening" thing internally?) What do you think about moving this stuff to a common utility file so both backends can use it, to avoid the duplication of code?

Hopefully this is better now? @sampsyo for visibility.

tests/backend/idl/dot-product.futil

Moved things into calyx-ir/utils. Things like get_mem_info and functions that get memories marked as external, along with their names

sampsyo

Wahoo! Looks great!! Go ahead and hit that green button whenever you are ready. 😃

rachitnigam · 2023-10-09T20:31:32Z

@nathanielnrn let's get this merged if everything looks good!

* WIP, add to backend options * add serde_json dependency for IDL backend * WIP work on IDL * Minimal working version of IDL generator Some open questions: Naming and placement. This is probably not xilinx specific so should be moved up a level into just src. Also, naming is generic and not good and should be changed TODO: Only works withs d1 memories, but should not be hard to alter * Let IDL generator handle multi-dim mems Previously could only handle 1 dimension mems. NOTE: This flattens dimensions into a single size variable that is spit out. However it may be useful to maintain info about shape of memory? Need to think about this more * add basic runt tests * clippy formatting * rename idl to yxi and make serde_json call shorter * add error conversion for serde_json for yxi backend * update runt tests for yxi * update cargo lock with serde_json * formatting * move yxi up a directory * WIP of extracting utility functions * Refactor toplevel and yxi util functions Moved things into calyx-ir/utils. Things like get_mem_info and functions that get memories marked as external, along with their names * clippy and formatting

nathanielnrn added 6 commits September 24, 2023 23:22

WIP, add to backend options

8b9ed2b

add serde_json dependency for IDL backend

2f968ce

WIP work on IDL

4658b7e

Let IDL generator handle multi-dim mems

2bcd8db

Previously could only handle 1 dimension mems. NOTE: This flattens dimensions into a single size variable that is spit out. However it may be useful to maintain info about shape of memory? Need to think about this more

add basic runt tests

bb9c4a7

nathanielnrn force-pushed the mem-interface-backend branch from 00988ef to 729ffdc Compare September 25, 2023 03:22

clippy formatting

5fce9bc

nathanielnrn force-pushed the mem-interface-backend branch from 729ffdc to 5fce9bc Compare September 25, 2023 03:24

nathanielnrn requested review from rachitnigam and sampsyo September 25, 2023 03:24

sampsyo reviewed Sep 29, 2023

View reviewed changes

nathanielnrn added 7 commits October 1, 2023 22:29

rename idl to yxi and make serde_json call shorter

6a7253b

add error conversion for serde_json for yxi backend

ccce931

update runt tests for yxi

6c1aa6c

update cargo lock with serde_json

af5e8be

formatting

148da2a

move yxi up a directory

8ea2726

WIP of extracting utility functions

98d9dcc

nathanielnrn mentioned this pull request Oct 4, 2023

Create an AXI-interface generator implemented Calyx #1733

Open

30 tasks

nathanielnrn added 2 commits October 7, 2023 00:00

Refactor toplevel and yxi util functions

d5dc9fb

Moved things into calyx-ir/utils. Things like get_mem_info and functions that get memories marked as external, along with their names

clippy and formatting

e04c1e7

nathanielnrn force-pushed the mem-interface-backend branch from 6814c63 to e04c1e7 Compare October 7, 2023 04:40

sampsyo approved these changes Oct 7, 2023

View reviewed changes

nathanielnrn merged commit 062b9a8 into master Oct 10, 2023
5 checks passed

nathanielnrn deleted the mem-interface-backend branch October 10, 2023 16:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create IDL Backend #1729

Create IDL Backend #1729

nathanielnrn commented Sep 25, 2023 •

edited

Loading

sampsyo left a comment

sampsyo Sep 29, 2023

nathanielnrn Oct 2, 2023

sampsyo Oct 2, 2023

sampsyo Sep 29, 2023

nathanielnrn Oct 7, 2023

sampsyo left a comment

rachitnigam commented Oct 9, 2023

Create IDL Backend #1729

Create IDL Backend #1729

Conversation

nathanielnrn commented Sep 25, 2023 • edited Loading

sampsyo left a comment

Choose a reason for hiding this comment

sampsyo Sep 29, 2023

Choose a reason for hiding this comment

nathanielnrn Oct 2, 2023

Choose a reason for hiding this comment

sampsyo Oct 2, 2023

Choose a reason for hiding this comment

sampsyo Sep 29, 2023

Choose a reason for hiding this comment

nathanielnrn Oct 7, 2023

Choose a reason for hiding this comment

sampsyo left a comment

Choose a reason for hiding this comment

rachitnigam commented Oct 9, 2023

nathanielnrn commented Sep 25, 2023 •

edited

Loading