CoT = learned search serialized on an append-only tape, per ML researcher
mblondel_ml · x · 2026-10-10
ML researcher Mathieu Blondel offers a crisp characterization of chain-of-thought: CoT = learned search, serialized onto an append-only tape.
- Learned: search cost is amortized at training; no explicit value function at inference — the same model generates and evaluates.
- Serialized: one sequence, so the search is mostly depth-first.
- Append-only: when backtracking, dead branches stay in context rather than being pruned.
A neat framing connecting CoT to classical search and explaining structural limits like costly backtracking.
More from AGI Musings
- If Claude were truly conscious, it wouldn't give itself a 15% chance of being so — inductionheads · 2026-10-10
- Model welfare is a ridiculous hill to die on while human suffering persists, says wolfie_ — emax · 2026-10-10
- Does heavy RL training break the 'LLMs are a blurry upload of humanity' intuition? — jd_pressman · 2026-10-10
- Why I wrote an NLP textbook in the age of AI that can teach anything — heiga_zen · 2026-10-10
- AI expands what you can explore, not your taste for what's worth exploring — srimisra · 2026-10-10
- Goodfire's Eric Bigelow on Forking Paths, Performative CoT, Reward Hacking — The Cognitive Revolution · 2026-10-10