Files

ch03

Rob Munger

Jan 27, 2025

1c1bc99 · Jan 27, 2025

This branch is 7 commits ahead of, 97 commits behind rasbt/LLMs-from-scratch:main.

Name		Name	Last commit message	Last commit date
parent directory ..
01_main-chapter-code		01_main-chapter-code	sdf	Jan 27, 2025
02_bonus_efficient-multihead-attention		02_bonus_efficient-multihead-attention	fixed typos (rasbt#414 )	Oct 24, 2024
03_understanding-buffers		03_understanding-buffers	Update README.md	Jul 30, 2024
README.md		README.md	Update bonus section formatting (rasbt#400 )	Oct 12, 2024

README.md

Chapter 3: Coding Attention Mechanisms

Main Chapter Code

01_main-chapter-code contains the main chapter code.

Bonus Materials

02_bonus_efficient-multihead-attention implements and compares different implementation variants of multihead-attention
03_understanding-buffers explains the idea behind PyTorch buffers, which are used to implement the causal attention mechanism in chapter 3