nanochat
The best ChatGPT $100 can buy
What it is
Karpathy's minimal, hackable, full-stack LLM pipeline: tokenizer, pretraining, finetuning, evaluation, inference, and a chat UI, training a GPT-2-class conversational model on one 8xH100 node in about 90 minutes for $50-100.
Why it's interesting
Demystifies the entire stack in readable code — and the community 'time-to-GPT-2' speedrun leaderboard turned reproducible pretraining into a sport, with the record around 1.65 hours.
Use cases
- Learning LLM training end to end
- Research baselines and ablation studies
- Teaching the full stack in one codebase
Who it's for
Students, researchers, and engineers who want to understand rather than just use
Setup
Moderate. One script — but you rent an 8xH100 node (~$24/hour) and bring $50-100 of cloud budget
Limitations & cautions
The resulting models are deliberately small and, in Karpathy's words, kindergarten-level. This is a school, not a factory.
Editorial takeaway
The most valuable output isn't the model — it's the mental model. Everyone who trains one stops being afraid of the stack.