BackendBench Examples

This repo hosts examples using BackendBench to perform model convergence testing.

Train nanoGPT using BackendBench with llm generated kernels

This example shows how to train nanoGPT using BackendBench with kernels generated by LLMs. The example is built upon @karpathy's nanoGPT project.

Prerequisites

uv venv
source .venv/bin/activate
git clone git@github.com:meta-pytorch/BackendBench.git
uv pip install BackendBench/
uv pip install matplotlib

Run

Enter the nanoGPT folder.

cd nanoGPT

Prepare the data.

cd data/shakespeare_char && uv run python prepare.py && cd ../..

Then run bash run.sh to train nanoGPT using the example kernels in generated_kernels_opinfo.

Loss curve and forwarding time

The training loss curve and forwarding time figures will be saved in figs/ folder.