Neural Networks: Zero to Hero — Code-Along

My notes and code following Andrej Karpathy's "Neural Networks: Zero to Hero" (YouTube playlist).

Each lecture gets its own folder under notebooks/ with the code I wrote along with the video, plus any notes on what I learned.

Setup

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
jupyter lab

Progress

• 1. The spelled-out intro to neural networks and backpropagation (micrograd)
• 2. The spelled-out intro to language modeling (makemore)
• 3. Building makemore Part 2: MLP
• 4. Building makemore Part 3: Activations & Gradients, BatchNorm
• 5. Building makemore Part 4: Becoming a Backprop Ninja
• 6. Building makemore Part 5: Building a WaveNet
• 7. Let's build GPT: from scratch, in code, spelled out
• 8. Let's build the GPT Tokenizer
• 9. Let's reproduce GPT-2