Feedback on Chapter 4 (DNN Architectures)

[zishenwan]

Some brief comments on Chapter 4 (DNN Architectures):

• Visual plots: I feel adding more visual plots would be helpful, e.g., add a RNN structure plot in Sec.4.4.2; add a attention structure plot in Sec.4.5.2; add a Transformer structure plot in Sec.4.6.4. Some visualization tool links might also helpful (e.g., CNN visualization, Transformer visualization). Those visualization can give more intuitions alongside existing texts and equations

• Sec.4.7:

  • Introduce roofline model concept might be helpful here, as it integrates compute/memory/data movement factors to offer insights on system-level design
  • Table.4.4: there's a cell missing in table (key challenges of sequential access), maybe can fill "access latency" (delays due to memory hierarchy or insufficient prefetching mechanisms)

[profvjreddi]

Thanks @zishenwan for the constructive feedback 🙏

Those visualization elements you linked in are great. I wasn't aware of them! Please do let me know if there are others.