Neural Networks Are Already Clean

The Core Thesis: The phenomenon of polysemanticity observed in early "model organisms" was not caused by superposition, but rather by a few standard, solvable issues. We can now interpret the majority of neurons in models reaching trillions of parameters, suggesting that the future of mechanistic interpretability is far brighter than previously assumed.

---

📌 The Executive Summary

After an extensive period analyzing circuits and neurons within language and image models, I have reached a conclusion: a network's internal mechanisms are its own legible building blocks.

For a long time, the field believed that networks "packed" concepts arbitrarily—a concept known as superposition. However, I believe this evidence was misinterpreted. What we thought was arbitrary packing is actually a pattern I call recursive generalization, which is far less problematic.

Consequently, we can study networks directly rather than spending our time trying to "untangle" them first.

---

🛠 Two Philosophies of Interpretability

There are two primary methodologies for approaching mechanistic interpretability:

The field initially started with the first approach but pivoted to the second because neural networks appeared to be tangled knots. This was driven by the discovery of polysemantic neurons—neurons that seem to perform multiple, unrelated tasks.

The "Knot" Hypothesis: Superposition

The prevailing theory was superposition. The logic followed this path:

1. A network runs out of available neurons.
2. It stops grouping related concepts.
3. It begins "folding" unrelated concepts together because they don't interfere with one another.

This implies that the specific grouping of concepts in a neuron is arbitrary, meaning the original network is illegible. To solve this, researchers use "decompositions" (like sparse autoencoders) to simulate what the network would look like if it were "unfolded."

---

🔍 Debunking the Superposition Myth

1. The Case of InceptionV1

Early analysis of InceptionV1 suggested its final layer was a chaotic mixture of arbitrary features. However, this was a failure of tools and early architecture, not a fundamental law of learning.

• The Old View: Almost no neurons in the last layer were interpretable.
• The New View: With modern architectural improvements (without changing data or size), 99-100% of neurons in that last layer are now easily interpretable.

I tested this across a massive scale: $\text{Scaling Range} \approx 300\times$

Observations on Scale:

• Small Models: Neurons cover coarse-grained concepts (e.g., one neuron for "all cars").
• Large Models: That single neuron differentiates into specific concepts (e.g., separate neurons for sedans, SUVs, and trucks).

The network doesn't "fold" unrelated things when space is tight; it simply generalizes more broadly.

2. The "Car-Dog" Circuit

In 2019, a famous example showed a "car" feature spreading into "dog" neurons. This was cited as proof of superposition (hiding a feature in a high-dimensional space).

Upon closer inspection, this wasn't "hiding"—it was simply ordinary generalization and composition. To date, no other circuit has been published that truly fits the "superposition" shape.

---

🤖 Modern Language Models (LLMs)

Early reports on LLMs were pessimistic. For example:

• Anthropic (2023): The median neuron scored $0/14$ on their interpretability rubric.
• OpenAI (2023): Only $\approx 1.7\%$ of neurons were deemed interpretable.

However, using better tools on modern models like Qwen3-32B, the data looks vastly different:

• ~90% of neuron poles: Have one primary job (with minor related offshoots).
• ~7% of neuron poles: Have two related jobs.
• ~2-4% of neuron poles: Remain misunderstood.

Summary of Neuron Distribution

# Conceptual representation of neuron clarity in modern models
neuron_stats = {
    "monosemantic": 0.90, # One main job
    "bisemantic_related": 0.07, # Two related jobs
    "unknown": 0.03 # Truly polysemantic/mysterious
}

Final Thought

The "tangled knot" of the neural network was largely an illusion created by early learning failures and primitive analysis tools. In reality, the networks are clean, and the path to understanding them is a straight line, not a puzzle to be solved.

Figure 1: Conceptual shift from viewing networks as arbitrary knots to seeing them as legible circuits.