Counterfactual token generation paper cited in stateless-LLM debate: near-zero-cost what-if reasoning for any LLM
gerardsans · x · 2026-09-05
Extending the "models have no self" debate, gerardsans cited an arXiv paper noting LLMs are stateless and can't reason about counterfactual alternatives to past tokens (e.g., rewriting a story if a different protagonist name had been sampled).
The paper's fix: a causal model of token generation built on the Gumbel-Max structural causal model lets any LLM perform counterfactual token generation at almost no extra cost — trivial to implement, no fine-tuning or prompt engineering — demonstrated on Llama 3 8B-Instruct and Ministral-8B-Instruct.
More from Research
- GPT-6 Astra hits 66% on ARC-AGI-3, near-100% with custom harness at ~$360 per game — AccBalanced · 2026-09-05
- Ambient Diffusion Policy accepted to CoRL 2026: training on suboptimal robot data — giannis_daras · 2026-09-05
- From 4 to 285 TFLOP/s: Part 1 of writing speed-of-light GEMM kernels on Blackwell B200 — HanGuo97 · 2026-09-05
- Research Roundup: Multi-Agent Coding Makes Results Worse — Use Single Agents for Write Tasks — PilgrimofHaqq2 · 2026-09-05
- Will interpretability ever be "solved"? A researcher argues probably not — burny_tech · 2026-09-05
- Intern Releases Lumina U2: Diffusion LLM Unifying Video, 3D Understanding and Image Generation — bdsqlsz · 2026-09-05