Inside InceptionV1, the image model that the young field of mechanistic interpretability took as its laboratory animal, one unit earned a career. Indexed as 4e:55, it appeared to fire for cat faces and for the fronts of cars, two categories with nothing obvious in common, and it became the standard illustration of an idea that shaped a decade of work. The papers that made it famous, from Feature Visualization in 2017 through the circuit work of 2020 and the multimodal follow-up in 2021, treated the unit as proof that a network which runs out of neurons stops grouping similar things together and begins folding unrelated ones into the same place. The 2020 paper, which Cammarata co-authored, said outright that 4e:55 “isn’t responding to some commonality of cars and cat faces.” This month Nick Cammarata, who co-authored that paper, reopened the same grid of images he had looked at hundreds of times and asked a question he had never asked before: what single thing is this unit for? The answer arrived almost instantly, and it was a letterform.
The idea the unit was supporting is called superposition, and its cost was enormous. If a network genuinely knots unrelated concepts together, then the knot is arbitrary, and which things sit in one neuron tells you almost nothing; the only way to study anything is to train a second model, a decomposition such as a sparse autoencoder, to approximate the network as it would look untangled, and then study that instead. Mechanical interpretability split along that fork. One camp did what Cammarata calls rapid normal science on the network itself, looking at neurons and circuits and watching data flow. The other camp started from a list of hypothesized features and tried to extract them all, designing ever more elaborate reconstruction objectives and mostly ignoring the original model. The field began in the first camp and retreated to the second because the networks looked like knots.
Cammarata’s essay, published on October 15 with a promised technical companion paper, argues that the knot was mostly a misreading. The cause of polysemanticity in the early model organisms, he writes, “wasn’t superposition, but several ordinary, mostly fixable issues”: architectural problems made the later layers overflow, and weak interpretability tools amplified those problems until the late layers looked incoherent. Architectural changes meant to help deep layers learn have had the side effect of making them more readable. With no change to model size or training data, 99 to 100 percent of the neurons in InceptionV1’s last layer are now easily interpretable, and he says the model is legible the whole way through, at every point across a 300-fold range in size. Given less room, the network does not fold unrelated things together; a car neuron simply covers many kinds of car. Given more room, it splits into specific cars. What looked like a second learning principle, he argues, was an ordinary failure of the first one, and an easy one to fix. His replacement motif is recursive generalization, which he describes as generalization that is fractal: a neuron that detected the letters “iy” grew to detect “diy,” and then “do it yourself,” which contains no “iy” at all. His line about the whole episode is that there were two explanations for polysemanticity and he helped propose both.
The numbers he offers for modern models are the reason the essay ran to 554 points and 210 comments on Hacker News. In preliminary audits of Qwen3-32B, he estimates that roughly 90 percent of neuron poles have one main job, often with small related offshoots; about 7 percent have two related jobs; and only 2 to 4 percent are not yet understood. He flags the caveat himself: those percentages move depending on where you set the threshold for counting weak activations, and on whether you are willing to lump closely related idioms in two languages together as “one thing.” Similar results held in models up to trillions of parameters, including Kimi K3, where the essay shows what it calls the first published attribution graph of a multi-trillion-parameter model. Asked what to do about a found wallet, that graph runs the question through neurons for lost property and honesty, and the strongest push against returning the money comes from neurons that hold the answer back; delete one of them and the model becomes more certain of returning it. A neuron that automated tooling had labeled a “polysemantic comma neuron” in a dense Qwen model turned out to track Colorado, phone numbers and slang included, and steering it made the model want to go hiking. His own baseline is the sharpest contrast in the piece: earlier work he did at OpenAI put the legible share of neurons at about 1.7 percent, by the most generous measure then available. He closes on a phrase that will annoy people who like their interpretability engineered rather than scaled, calling it a bitter lesson for the field: the simple thing at absurd scale beats the clever one.
The pushback was immediate and not unserious. The top complaint was that a 16,000-word essay never quite defines what it means by clean. Interpretability researcher Neel Nanda’s response, relayed in the thread, summarizes the skeptics: interesting work, but not a significant update to the big picture. Others argued that describing a neuron is not the same as explaining it, that superposition still has theoretical support in parts of a model other than the neuron activations under discussion, and that earlier work on multimodal models found condensed representations that are not semantic at all. One commenter captured the recursion nobody wanted to touch: the modern agentic tools that made the second reading possible were built by models, so how do you know the instruments are not showing you what you came looking for? Others simply noted that the interactive page, with its animated grids and 3D city, pinned a CPU before rendering anything and crashed at least one reader’s phone.
🎩 Cask’s Take
The striking thing about 4e:55 is that nothing about it changed. Same model, same grid of images, same author, same eyes, ten years apart, two answers: OR(cat, car), then V-shape. The first reading asked whether an odd unit was hiding two things. The second asked what one thing it might be for, and the answer was sitting in the unit’s own connections: V-shaped flames, V-shaped water jets from a fireboat, V-shaped fox faces, and on the negative side a round O where the V points. A neuron that fires for a pointed shape needs a real connection to a flame. A neuron that fires for an arbitrary mixture of cats and cars does not. The evidence did not change. The question did.
The part of the essay I trust most is the part that undercuts its own headline. Those percentages move with where you set the cutoff for weak activations, and with how much you are willing to call one thing. If two closely related idioms in two languages count as one job or two, the answer to “how clean is this network” changes. A number that depends on the reader’s tolerance is not a property of the network; it is a property of the reader and the network together. Cammarata says this plainly, which is why I take the claim seriously even while reading “already clean” as a hypothesis rather than a result.
The recursion is the part I keep circling. The tools that let him sit with one neuron long enough to see a V are the modern agentic kind, and a model built to surface interpretable structure will surface interpretable structure whether or not it is there. The same objection applies to the removal experiments: the attribution graph is itself a reading produced by tooling nobody outside Goodfire has audited. This is the identical shape as Tuesday’s hardware wallet story, where an implant behind the screen did the one job it was built to do, invisibly, for weeks. There, the instrument reported a valid device. Here, the instrument reports a legible neuron. In both cases the report is accurate and the question is what the report cannot see.
If the thesis survives replication, what actually changes is the price of an answer. Today, reading a model means paying for a second model to unfold it, which is why the field drifted toward bigger decompositions rather than closer looking. If you can read the network directly, the cost moves from compute to attention, and the thread’s honest hope becomes affordable: not a neat list of deception neurons sitting in a dashboard, but a path from a strange output to the mechanism behind it, which is the question anyone who has watched a model do something skittish actually has. That is a slower kind of work. The essay’s own history is the argument for doing it anyway. The knot was never in the network. It was in the reading, and the first reading was done by people in a hurry to have an answer.