At a glance
Chris Olah has spent a decade arguing that neural networks are objects you can look inside, not black boxes you can only poke from outside. This talk, given at an Alignment Workshop hosted by FAR.AI, is his compact case for mechanistic interpretability: the program of reverse engineering the computation a trained network actually performs, in terms of features and the circuits that connect them.
The story has three beats. First, the good news from vision models: real, nameable features exist inside networks, and the circuits that build complex features out of simple ones can be read off the weights. Second, the obstacle: most neurons are polysemantic, firing for several unrelated things, which breaks the neat picture. Third, the explanation and the way through: superposition, the hypothesis that models represent far more features than they have dimensions by placing sparse features along nearly orthogonal directions, and dictionary learning as the tool that pulls those features back apart.
The framing throughout is scientific rather than engineering. Olah talks about interpretability the way a biologist talks about microscopy, and the payoff he cares about is auditing: finding out what a model is doing rather than asking it and hoping the answer is honest.
The deep explanation
Features and circuits
The foundational claim, developed in the Circuits thread on Distill, is that networks decompose into two kinds of thing. Features are the directions in activation space that correspond to something meaningful: a curve at a particular orientation, a dog head, a piece of Python syntax. Circuits are the weighted connections between features across layers, and they are readable as algorithms.
The worked example from vision is curve detectors in InceptionV1. There are neurons that respond to curves at specific orientations, and you can verify this several ways: by finding the images that maximize them, by feature visualization, by synthetic curve stimuli, and by reading the weights. Then you can see the next layer build on them. Combine curve detectors in a particular arrangement and you get circle detectors. Combine those and you get more complex shapes. Olah's point is that this is not a story imposed on the weights; it is visible in the weights, and it reproduces.
Two claims come out of this that shape the whole program. Features are the right unit of analysis, more so than neurons or layers. And there is evidence for universality: the same features, curve detectors, high low frequency detectors, and others, appear across different architectures trained on different data, which suggests they are natural solutions to the problem rather than accidents of one training run.
The problem: neurons are polysemantic
The awkward finding is that if you go looking for a clean neuron per concept, most of the time you do not find it. A single neuron will respond to cat faces and car fronts and something else entirely. This is polysemanticity, and it wrecks the simple version of the interpretability program, because you cannot narrate a computation whose variables each mean three unrelated things.
Olah's response is not to abandon the features picture but to ask why a network would build such a thing. The answer he proposes is that polysemantic neurons are not the model's real variables. They are shadows of a larger set of variables projected into too few dimensions.
Superposition
The superposition hypothesis is the centerpiece. It says a network wants to represent many more features than it has dimensions, and it can get away with doing so because features are sparse: in any given input, only a small fraction of all possible features are actually present.
The geometry is the intuition. In a space of n dimensions there are only n perfectly orthogonal directions, but there are exponentially many directions that are nearly orthogonal. If features rarely co occur, you can assign each one a nearly orthogonal direction and read it back with only a small amount of interference from the others. The network is running something very close to compressed sensing, and it is a good trade: a little noise in exchange for representing far more concepts than dimensions.
The toy models work makes this concrete. Train a small network to compress and reconstruct sparse features and you can watch superposition switch on as sparsity increases. At low sparsity the model keeps the most important features orthogonal and drops the rest. As sparsity rises it starts packing features together in structured geometric arrangements, and the arrangements it chooses turn out to be recognizable polytopes. Superposition is not a pathology, it is the optimal thing to do under those constraints.
That also reframes polysemanticity. A neuron fires for several unrelated things because it is one axis of a space in which many more features than axes have been packed. The neuron was never the unit; the feature direction is.
Dictionary learning: getting the features back
If superposition is the reason features are hard to read, then the fix is to undo the projection. That is a dictionary learning problem: find an overcomplete set of directions such that every activation vector is a sparse combination of a few of them.
In practice this is done with sparse autoencoders, trained to reconstruct a layer's activations through a much wider hidden layer under a sparsity penalty. The result, published as Towards Monosemanticity, is that the recovered directions are dramatically more interpretable than the raw neurons. Features emerge that correspond to specific and nameable things: Arabic script, DNA sequences, base64 encoded text, legal language, particular syntactic positions in code. Each fires for one thing rather than four, which is exactly what the superposition story predicted would happen once you decompressed properly.
Olah is careful about what this does and does not establish. The features are more interpretable, not perfectly interpretable. The number of features you recover depends on the size of the dictionary, and there is a long tail. And going from a catalogue of features to a full circuit level account of a large model is a scaling problem that was, at the time of this talk, unsolved.
Why this matters for safety
The motivation is auditing. Every other approach to knowing whether a model is safe consists of observing behavior: test it, prompt it, red team it. That is empirical evidence about the cases you tried. It cannot rule out behavior in cases you did not think to try, and it is exactly the wrong tool against a model that behaves differently when it believes it is being evaluated.
Interpretability is the only approach that examines the mechanism rather than the output. If you can find the features and read the circuits, you can ask questions the behavioral tests cannot: does this model represent a concept of being observed, is there machinery here for withholding information, is this refusal implemented by understanding or by keyword matching. Olah's analogy is the microscope. Biology did not become a science by testing organisms behaviorally; it became one when instruments let people see the parts.
He is honest about the state of play. The field is young, most of its results come from small models or narrow slices of large ones, and there is no guarantee the program scales. But he argues the alternative, deploying systems of increasing capability with no way to inspect them, is not a research strategy.
Key takeaways
- Features are the unit of analysis inside a network, not neurons and not layers. Circuits are the readable weighted connections between features.
- Curve detectors and the circuits built on them in vision models are the existence proof: real, verifiable, reproducible structure inside trained weights.
- Universality is a genuine finding. The same features recur across architectures and training runs, which suggests they are natural solutions rather than accidents.
- Most neurons are polysemantic, which breaks the naive one neuron one concept picture.
- Superposition explains why: sparse features get packed into nearly orthogonal directions, so a model represents far more features than it has dimensions, accepting a little interference in return.
- Dictionary learning with sparse autoencoders recovers monosemantic features from that packing, producing directions that correspond to nameable things.
- The safety case is auditing. Behavioral testing tells you about the cases you tried; mechanistic interpretability is the only route to statements about what a model is actually doing.
Where this sits in the LLM Learning track
This is the second half of the behavior part of the track. The reinforcement learning talk before it shapes a model from the outside by rewarding what you want. This one asks whether you can look inside and check. Taken together they are the two available strategies for making a system trustworthy, and the honest summary is that the first is deployed everywhere and imprecise, while the second is precise and not yet at scale.
Resources mentioned
- Chris Olah's blog, including the classic posts on understanding LSTMs and neural network visualization
- Zoom In: An Introduction to Circuits and the rest of the Circuits thread on Distill
- Curve Detectors
- Toy Models of Superposition
- Towards Monosemanticity: Decomposing Language Models With Dictionary Learning
- A Mathematical Framework for Transformer Circuits and In-context Learning and Induction Heads
- Transformer Circuits, where this line of work is published
- FAR.AI and the Alignment Workshop series that hosted the talk
About this page
This reconstruction was built from the public record of the talk and the published research it draws on rather than from its caption track, because the machine that assembled it had no network route to YouTube. The argument and its structure are here; the verbatim transcript, the timestamped chapter map, and pull quotes are not, and those get backfilled on the next run from a machine that can reach the source. Until then, watch it on YouTube alongside this page.
