LLM Learning
Twelve talks, in order, that take you from what a large language model is to shipping something built on one.
This is the first learning track on the site: a curriculum rather than a shelf. Twelve of the best explanations anyone has published about large language models, put in the order that actually teaches the subject, with a note on why each one earns its slot. You can read the pages straight through as a book, or use each page as a study guide beside the video. Nothing here is a hot take or a product launch. Every one of these was chosen because it is still worth an hour of your attention long after it was published.
Get the mental model
What a large language model is, what happens inside it, and which parts of it can actually be measured. Three passes over the same territory at increasing resolution.
-
1
Andrej Karpathy Deep Dive into LLMs like ChatGPT
The whole stack in one sitting: pretraining, tokenization, fine tuning, reinforcement learning, then the cognitive quirks that follow from all of it. Start here even if you think you know this material, because everything later in the track attaches to something in this video.
-
2
3Blue1Brown Attention in transformers, step-by-step | Deep Learning Chapter 6
Karpathy treats the transformer as a machine. This opens the machine and walks through attention one matrix at a time, with a running parameter count against real GPT-3 dimensions so nothing floats free of the numbers.
-
3
Sasha Rush Large Language Models in Five Formulas
The quantitative frame. Five formulas covering generation, memory, efficiency, scaling, and reasoning, each with an honest statement of what it fails to explain. This is the page to reread whenever a claim about LLMs sounds too confident.
Build one from scratch
Nothing replaces having typed it. Two long builds, in the order the pipeline actually runs: first the tokenizer, then the model.
-
4
Andrej Karpathy Let's build the GPT Tokenizer
The first step in the pipeline and the source of a startling share of LLM weirdness. Build byte pair encoding from Unicode up, then watch the spelling, arithmetic, and non English failures stop being mysterious.
-
5
Andrej Karpathy Let's reproduce GPT-2 (124M)
Four hours from an empty file to a trained model that beats the published GPT-2 124M numbers. Writing the architecture is the quick part; the middle two hours on making it fast are where the real craft is.
Make it behave
A trained model is not yet a usable one. Two talks on the only two strategies available: shape the behavior from outside with rewards, or look inside and find out what it is doing.
-
6
Berkeley EECS John Schulman - Reinforcement Learning from Human Feedback: Progress and Challenges
The clearest argument anywhere for why imitation training teaches a model to hallucinate and why reinforcement learning is the lever that can teach it to hedge instead. From the person who led the work.
-
7
FAR.AI Chris Olah - Looking Inside Neural Networks with Mechanistic Interpretability
The other strategy. Features, circuits, and the superposition hypothesis that explains why individual neurons mean several things at once. Behavioral testing tells you about the cases you tried; this is the only route to statements about what a model actually does.
Ship something with it
The engineering half. What to build, how to tell whether it works, and when to hand control to the model. This is the part that decides whether a demo becomes a product.
-
8
Jeremy Howard A Hackers' Guide to Language Models
The practical ladder: prompt the strongest model, add function calling, run open models locally, add retrieval, and only then fine tune. Code first throughout, and it ends on the mistake almost everyone makes about their own evaluation.
-
9
AI Engineer How to Construct Domain Specific LLM Evaluation Systems: Hamel Husain and Emil Sedgh
The answer to that mistake, and the least glamorous, most load bearing hour in the track. Assertions in CI, human review with an aligned LLM judge, then production. Build your own trace viewer; friction in looking at data is what stops teams from improving.
-
10
LangChain Building Effective Agents with LangGraph
Every pattern from Anthropic's Building Effective Agents implemented in code, ordered by how much control you hand to the model. The useful part is the boundary: reach for a workflow first, and build an agent only when the path genuinely cannot be written down.
Where it goes next
Two talks about the open questions, one from the research side and one from the building side. Watch these last, when you have enough context to judge them.
-
11
Ilya Sutskever Ilya Sutskever: Sequence to sequence learning with neural networks: what a decade
A decade of scaling reviewed by the person who wrote its opening chapter, and the line the talk is remembered for: pretraining as we know it will end, because compute keeps growing and data does not.
-
12
Y Combinator Andrej Karpathy: Software Is Changing (Again)
The closer, and the only video here aimed at someone deciding what to build. Software 1.0, 2.0, and 3.0, the LLM as an operating system, and a practical checklist for partial autonomy products. It ends on the decade of agents rather than the year of them.
