At a glance
Jeremy Howard built fast.ai on the belief that you learn this material by running code, not by reading about it, and this talk is that philosophy applied to language models. It is a code first tour aimed at people who want to build something today: what a language model is and how the training recipe he helped popularize with ULMFiT became the standard, how to actually get value out of the strongest hosted models, then the engineering ladder of API use and function calling, running open models on your own hardware, retrieval, and fine tuning a small model on a narrow task.
The tone is unusually grounded. Howard spends real time on what these models cannot do and on how easy it is to fool yourself about what they can, and he is blunt that most people using them are using them badly, mainly by not iterating on the prompt and not checking the output against anything.
Everything he demonstrates runs in a notebook, and the notebooks are public, which is the point.
The deep explanation
What a language model is, and the three stage recipe
Howard starts where the field started for him. In 2018 ULMFiT showed that you could pretrain a language model on a general corpus, fine tune that model on the text of your target domain, and only then train a classifier on top, and that this three stage transfer learning approach beat training from scratch by a wide margin. That structure, general pretraining then task adaptation, is the ancestor of everything current, which he notes with the calm of someone who watched it become an industry.
Applied to modern models, the stages are pretraining on a very large corpus, instruction tuning on examples of following instructions, and then a preference stage that shapes behavior. His framing of the base objective is the useful one for builders: predicting the next token sounds trivial, but to predict well over arbitrary text you have to model an enormous amount about the world, because the text is about the world. That is the argument for why the objective produces something more general than autocomplete.
He also does the useful demystification on the units. Tokens are word fragments; the model emits a distribution over them; you sample; you feed it back. Nothing about that loop is intelligent by itself, and yet the loop is all there is.
Using the strongest model well
A large part of the talk is spent on getting real work out of hosted models, and Howard treats this as a skill rather than a formality:
- Use the best available model. He is direct that at the time there was a substantial capability gap between the frontier model and everything else, and that people forming opinions about what language models can do based on weaker models are forming opinions about the wrong thing.
- Custom instructions matter. Telling the model who it is talking to and what standard of answer you want changes output quality more than most prompt tricks.
- The model is not a search engine and it is not a database. Ask it to reason over text you supply, and it will do well. Ask it to recall a specific fact from its weights, and you are rolling dice.
- Check the output. In a domain you know, the errors are obvious. In a domain you do not, they are invisible, which is precisely why people overestimate the models in unfamiliar areas.
He also walks through the practical economics of the API: pricing per token, rate limits, and the fact that the cost of a serious experiment is usually much lower than people expect, which removes the main reason people avoid iterating.
Function calling and building a code interpreter
The step from chat to system is function calling. You describe the functions available, the model returns a structured request to call one with arguments, your code executes it and hands back the result. Howard builds this up live, and the demonstration people remember is a small code interpreter: give the model the ability to run Python, let it write code to answer a question, execute it, and return the output.
Two things come out of this that matter more than the code:
- This is how you fix arithmetic and data work. The model is a poor calculator and a good programmer, so route the calculation through code.
- Executing model written code is an actual security problem. He does not gloss over it. Sandboxing is not optional if any input is untrusted.
Running open models yourself
The next rung is running models locally, which Howard treats as both practically useful and pedagogically important, because it forces you to understand memory, precision, and what a checkpoint actually is.
The pieces he covers:
- The Hugging Face ecosystem as the default source of weights and the standard loading code.
- Quantization, which stores weights at reduced precision so a model that would not fit in your GPU memory does. This is the single technique that makes local inference practical on consumer hardware, at a modest and usually acceptable quality cost.
- The instruction format matters. Open instruction tuned models expect their prompts in a specific template, and getting the template wrong quietly degrades output in ways that look like the model being bad.
- What you get and what you give up. Local models are private, free per call, and available offline. They are also weaker, and the gap is largest exactly where you need reliability.
Retrieval augmented generation
Rather than presenting retrieval augmented generation as a product category, Howard builds the simplest possible version and shows that it works: embed a corpus of documents with a sentence embedding model, embed the question, find the nearest documents, paste them into the prompt, ask the question again with that context attached.
The insight he wants you to take is that this is not complicated, and that most of the quality comes from the retrieval step rather than from anything clever in the prompt. If the right passage is in the context, a decent model will use it. If it is not, no amount of prompt engineering rescues the answer. That reframes retrieval work as an information retrieval problem, which has decades of existing technique behind it.
Fine tuning a small model on a narrow task
The final rung is fine tuning, and Howard picks a deliberately unglamorous example: teaching a small open model to write SQL queries against a schema. The point of the example is the shape of the result. A small model fine tuned on one narrow task can match or beat a much larger general model at that task, at a fraction of the inference cost, while remaining useless at everything else.
That is the trade every builder should understand:
| Approach | Costs you | Wins when |
|---|---|---|
| Prompt the frontier model | Per call price, latency, data leaves your machine | Almost always the right first move |
| Add retrieval | An index to build and maintain | The answer lives in documents you have |
| Function calling | Engineering and a sandbox | The task needs computation, data, or actions |
| Run an open model locally | Hardware and quality | Privacy, cost at volume, or offline use |
| Fine tune a small model | Data collection and a training loop | One narrow task, done constantly, where cost matters |
He is also clear about the sequencing. Fine tuning is the last resort, not the first, because prompting and retrieval solve most problems and require no training data. The reason to reach for it is economics or a task so specific that no prompt describes it well.
Keeping up
The closing advice is about the discipline rather than the tools. Read the papers that matter rather than the summaries of them. Run the notebook rather than reading it. And be suspicious of your own evaluation: the single most common failure Howard sees is a builder who is confident their system works because they tried it a few times on examples they chose themselves.
Key takeaways
- The three stage recipe from ULMFiT, general pretraining then domain adaptation then task training, is the ancestor of the modern pretrain, instruction tune, preference tune pipeline.
- Judge language models by the best available one. Opinions formed on weaker models are opinions about a different technology.
- The model is strong at reasoning over text you give it and unreliable at recalling facts from its weights. Design around that split.
- Function calling turns a chat model into a system. A code interpreter is the highest leverage function you can give it, and executing model written code needs a sandbox.
- Quantization is what makes local models practical, and getting the instruction template right matters more than people expect.
- Most of the quality in retrieval augmented generation comes from retrieval, not from prompting. It is an information retrieval problem.
- Fine tuning a small model can beat a much larger one on a single narrow task at a fraction of the cost, and it should be the last thing you try, not the first.
- The most common mistake is believing your system works because you tested it on examples you chose.
Where this sits in the LLM Learning track
This opens the building part of the track. The earlier parts explain what the model is and how it was made; from here on the question is what you do with one. Howard's closing warning about evaluating your own system on examples you picked yourself is the exact problem the next video in the track sets out to solve, so the two are best read back to back.
Resources mentioned
- fast.ai and Jeremy Howard's site, where the notebooks for this talk live
- Universal Language Model Fine-tuning for Text Classification, the ULMFiT paper
- Hugging Face and the transformers library
- The OpenAI API and its function calling documentation
- Llama 2 and the open model ecosystem it opened up
- Sentence Transformers for the embedding step of retrieval
- Kaggle notebooks and Colab as free places to run everything in the talk
About this page
This reconstruction was built from the public record of the talk rather than from its caption track, because the machine that assembled it had no network route to YouTube. The structure, the ladder of techniques, and the advice are here; the verbatim transcript, the timestamped chapter map, and pull quotes are not, and those get backfilled on the next run from a machine that can reach the source. Until then, watch it on YouTube with a notebook open.
