UChicago paper: LLMs can't anticipate their own cruxes — sampling many rollouts is unavoidable
rohanpaul_ai · x · 2026-10-24
A University of Chicago paper (arXiv 2610.21006) shows an LLM's a priori account of its decision cruxes barely matches what actually drives disagreement across its answers. In March Madness and Metaculus forecasting tests, predicted cruxes shared only 13.4% of clusters with real ones (below the 39.2% random baseline); a single rollout covered just 28.6%. Using a cheap reranker to compare many rollouts cuts compute 86x versus an LLM judge. Takeaway: for high-stakes decisions, sample many answers and inspect where they disagree.
Related event: UChicago Paper Finds LLMs Cannot Anticipate Their Own Cruxes(4 posts)→
More from Research
- Dessa's dialog-only voice training isn't new: dGSLM did it in 2022 with just 2k hours — rdesh26 · 2026-10-24
- NYU talk argues 30+ years of consciousness research may be partly wrong — birchlse · 2026-10-24
- Psycholinguistic analysis finds Claude Opus 5 the most out-of-distribution model tested — gleech · 2026-10-24
- Google Research tests 180 agent setups: multi-agent hurts sequential tasks by 39-70% but boosts parallel ones 80.9% — ombre-ai · 2026-10-24
- Loopholing: pushing self-conditioning all the way in discrete diffusion language models — NandoDF · 2026-10-24
- Review-it tool suggests 10 best-fit journals and conferences for your paper — Faheem_uh · 2026-10-24