The detective novel thought experiment: why hidden states beat tokens for model handoff
CShorten30 · x · 2026-09-15
A researcher shares a thought experiment on why hidden states matter more than tokens:
- Give a language model an entire detective novel ending with "the killer was" — to emit the next token, it must model who was where, which alibis conflict, and what the author planted in chapter three, all inside a single forward pass.
- But it only outputs a name. A second model continuing the story receives just that name — the whole case is gone.
- That's the situation of every multi-model system today: each model rebuilds the case from scratch. The author notes @mostikai is building a channel to hand over hidden states instead.
The analogy illustrates the information and compute waste of token-level handoffs and motivates inter-model state transfer research.
More from coding & agent
- Pyrefly 1.3 ships experimental tensor-shape checking for JAX, NumPy and Polars schemas — tdhopper · 2026-09-15
- Codex and Grok Bot share the same flaw: making users do the organizing — jesselyu · 2026-09-15
- Astra writes code humans can no longer read: 'machineslop' and reward hacking — jiqizhixin · 2026-09-15
- Read-only Shopify catalogue MCP server: stdlib-only, one file, with a 183-assertion self-test — PrestigiousAddress47 · 2026-09-15
- BlaBla: executable project memory cuts coding agent handoff cost by 63% vs prose context — KiborgikDEV · 2026-09-15
- Vercel AI SDK harness layer adds native subscription auth for Claude Code, Codex, Cursor and more — cramforce · 2026-09-15