Bonus Chapter: The Hidden Thread
This book has a thread that was never explicitly stated. Now it is time to speak it aloud.
1. That Wall
Chapter 3, word vectors did something magical: king minus man equals queen. This is association, not reasoning. But at the time we did not clearly state the difference.
Chapter 5 said: statistical correlation is not reasoning. A model having seen a million cats does not mean it knows what a cat is. But we only pointed out this problem, not where it comes from.
Chapter 6 gave the do-operator. Pearl said: observation is never enough, you need intervention. Cut the incoming edges of the causal graph, force a variable to take a value, then re-infer—this is causal inference, not conditional expectation.
The Chapter 9 bonus chapter discovered: self-attention is mathematically equivalent to one retrieval step of Hopfield associative memory. From 1982 to 2024, the entire lineage—classical Hopfield, modern Hopfield, self-attention, linear attention, SSM—all do the same thing: given a query, find the most relevant content in the memory bank, normalize, weight, and return.
Chapter 12's Yonglin Formula says: the reasoning chain ultimately converges back to the prior anchor. In Hopfield's language: the statistical bias of the training data is encoded as the global minimum of the energy function; the longer the reasoning chain, the more it is pulled into this attractor.
Now put these together:
What Transformer does is associative retrieval. Associative retrieval is
Throughout this entire book, this wall has always been there; we just never directly spoke its name.
2. What the Wall Is Made Of
Associative memory is passive. Give me a query, I find the most relevant memory, and return it to you. The mathematical core of this process is the normalized inner product—what softmax does is partition function normalization, nothing more.
Causal inference is active. I cut off your connections to other variables, force you to take a certain value, then ask: what is the world like now? This requires knowing the structure of the causal graph, and the structure of the causal graph cannot be read out from observational data—it can only be read out from interventions.
Where is the problem? What neural networks learn is
This is not a problem of insufficient training data. This is not a problem of insufficient model scale. This is a problem that the training objective itself is not causal inference.
Gödel tells us: any sufficiently strong formal system contains true propositions that it cannot prove. The root is self-reference—the system uses its own rules to verify itself, a gap necessarily exists.
The root of this wall is similar: associative memory uses correlation to approximate causation, but correlation and causation are mathematically different things; the former cannot grow into the latter. It is not that the approximation is not good enough; structurally, there is a missing layer.
3. How the Human Brain Crosses the Wall
The human brain is also weighted summation of neurons. Synaptic weights, activation functions, backpropagation—you can describe it with more or less the same mathematics.
But humans can do counterfactual reasoning. "If I had not said those words back then, how would he have reacted?" This question, within any purely associative memory framework, cannot even be asked, let alone answered. Associative retrieval can only ask "who is most similar to me," not "if the world were different, what would happen."
So how does the human brain do it?
We don't know. This is one of the questions neuroscience currently has no answer to. There are several conjectures:
Conjecture 1: The human brain has an additional causal modeling module on top of associative memory. The prefrontal cortex is responsible for planning and decision-making; it may maintain some kind of explicit causal graph structure and perform causal inference after associative retrieval is completed.
Conjecture 2: Counterfactual reasoning is a special editing operation on memory. It is not a different set of mathematics, but a directed modification of existing memories—replacing the event "I said those words back then" with "I did not say them," then letting the associative network re-evolve from the modified initial state. Mathematically, this is still associative retrieval, only operating on counterfactual initial conditions.
Conjecture 3: The human brain's training regime is fundamentally different from neural networks. Infants learn causal relationships by actively exploring the world—reaching out to grab things, knocking down blocks. This is real intervention, not passive observation. Perhaps the key to causal inference capability lies not in architecture but in the nature of training data: interventional data, not observational data.
These three conjectures are not mutually exclusive. Maybe all are right. Maybe all are wrong.
4. Why This Hidden Thread Matters
You can understand this book as two intertwined narratives:
The explicit thread: Where the boundary of AI reasoning capability lies. The halting problem, P/NP, the Yonglin Formula—these are known, formalizable boundaries.
The hidden thread: There is a deeper boundary, one that has not been formalized, not even named—the chasm between associative memory and causal inference. The entire edifice of modern machine learning, from the perceptron to the Transformer, is built on the foundation of associative memory. This foundation is immensely powerful, sufficient to support language understanding, image generation, protein folding prediction. But structurally, it cannot support genuine causal inference.
This is not a pessimistic conclusion. This is a clear diagnosis.
A clear diagnosis is better than vague optimism. When you know where the wall is, you can decide: to achieve excellence on this side of the wall, or to try building a new wall with different materials, or to search for a door.
Currently we are doing the first two things. The third thing—that door—we still do not know where it is.
5. The True Map of the Reasoning Kingdom
At the beginning of this book, we said we would draw a map of the Reasoning Kingdom. Chapter 13 drew one: at the center are P-class problems, the outer ring is undecidable problems, and the boundary is the phase transition region.
That map is accurate. But it only drew the dimension of computational complexity.
What the hidden thread reveals is another dimension: the dimension of reasoning depth.
In this dimension, reasoning is divided into three layers—Pearl's Causal Hierarchy:
First layer: Observation (seeing)—conditional expectation,
Second layer: Intervention (doing)—interventional distribution,
Third layer: Counterfactuals (imagining)—counterfactual distribution,
Transformer is locked by architecture at the first layer, can occasionally simulate the appearance of the second layer, and the third layer is currently forever closed to it.
The true map of the Reasoning Kingdom needs to annotate both the complexity dimension and the reasoning depth dimension simultaneously. On the first dimension we already have a fairly clear picture. The second dimension is still under exploration.
6. The Last Word
This book stops before the boundary. It gives no solution, because there is no solution.
But stopping before the boundary, and pretending the boundary does not exist, are two completely different things.
You now know where that wall is. You know what associative memory can do and cannot do. You know why the do-operator is absent in existing architectures, and you know this is not an implementation problem but an ontological problem. You know that the convergence of the Yonglin Formula is the inevitability of the energy attractor, and you know that Gödel's incompleteness and it are projections of the same thing in different languages.
This is not knowledge to make you despair. This is knowledge to let you know where it is worth continuing to dig.
That door, perhaps it is in someone's doctoral thesis. Perhaps in a question that has not yet been asked. Perhaps it requires entirely different mathematics.
The boundary of the Reasoning Kingdom is not the end.
It is the starting point of the next journey.