Files

ch04

Rob Munger

Jan 26, 2025

e291e72 · Jan 26, 2025

This branch is 7 commits ahead of, 97 commits behind rasbt/LLMs-from-scratch:main.

Name		Name	Last commit message	Last commit date
parent directory ..
01_main-chapter-code		01_main-chapter-code	sdf	Jan 26, 2025
02_performance-analysis		02_performance-analysis	update card	Oct 11, 2024
README.md		README.md	Update bonus section formatting (rasbt#400 )	Oct 12, 2024

README.md

Chapter 4: Implementing a GPT Model from Scratch to Generate Text

Main Chapter Code

01_main-chapter-code contains the main chapter code.

Bonus Materials

02_performance-analysis contains optional code analyzing the performance of the GPT model(s) implemented in the main chapter
ch05/07_gpt_to_llama contains a step-by-step guide for converting a GPT architecture implementation to Llama 3.2 and loads pretrained weights from Meta AI (it might be interesting to look at alternative architectures after completing chapter 4, but you can also save that for after reading chapter 5)