This repository contains the notebooks related to hardware-aware training of spiking neural networks presented and their deployment onto FPGAs at ISFPGA 2024 (Monterey, CA) for the workshop Who needs neuromorphic hardware? Deploying SNNs to FPGAs via HLS co-presented by Jason Eshraghian and Fabrizio Ottati.
How can we use natural intelligence to improve artificial intelligence? The human brain is a great place to look to improve modern neural networks and reduce their exorbitant energy costs. While we may be far from having a complete understanding of the brain, we are at a point where a set of design principles have enabled us to build potentially more efficient deep learning tools. Most of these are linked back to spiking neural networks (SNNs). In a cruel twist of irony, the neuromorphic hardware that is out there for research and/or commercial use, is considerably more expensive (and often less performant), and harder to obtain than a consumer-grade GPU. How can we move towards using low-cost hardware that sits on our desk, or fits in a PCIe slot in our desktops, and accelerates SNNs? FPGAs might be the solution. This tutorial will take a hands-on approach to learning how to train SNNs for hardware deployment on conventional GPUs, and running these models on a embedded class FPGA (AMD Kria KV260) for inference. FPGA inference is achieved using high level synthesis, employing the AMD Vitis HLS compiler, and using a dataflow architecture1 of a deep SNN, with in-hardware testing.
There are two tutorial components in this repo:
- One for hardware-aware training spiking neural networks using snnTorch
- One for the bitstream, PYNQ and C++ code to run the HLS model on the FPGA.
| Title | Colab Link |
|---|---|
| Hardware-Aware Training of Spiking Neural Networks with snnTorch |  |
| Cheat-Sheet: Hardware-Aware Training of Spiking Neural Networks with snnTorch | |
In the hw folder you will find:
- the bistream of the accelerator, implemented for the Kria KV260 AI starter kit:
design_1.bit. - the PYNQ script to execute inference:
main.pyand some utilsutils.py. - the addresses configuration used to communicate between the Zynq PS and the HLS IP:
xwrapper_hw.h. - the hardware handoff file
design_1.hwh.
This material will be provided ASAP. The HLS C++ code will be released in the upcoming weeks.
Jason K. Eshraghian received the B.Eng. (electrical and electronic), L.L.B., and Ph.D. degrees from The University of Western Australia, Perth, WA, Australia, in 2016 and 2019, respectively. From 2019 to 2022, he was a Post-Doctoral Research Fellow at the University of Michigan, Ann Arbor MI, USA. He is currently an Assistant Professor with the Department of Electrical and Computer Engineering, The University of California at Santa Cruz, Santa Cruz, CA, USA. His research interests include neuromorphic computing, resistive random access memory (RRAM) circuits, and spiking neural networks.
Fabrizio received his Ph.D. from Politecnico di Torino in 2024, with a thesis on efficient inference of spiking neural networks on FPGA platforms. He is now an AI Computer Architect at NXP semiconductors. He is mainly interested in deep learning and computer architecture, beyond trying to learn how to write decent code.
Footnotes
-
Work carried out by Fabrizio Ottati during the Ph.D. at Politecnico di Torino. ↩