New Paper: LLM Forecasts Get More Incoherent as Logical Relations Between Events Increase
soumitrashukla9 · x · 2026-09-05
A new paper by Suproteem Sarkar and Isaiah Andrews measures probabilistic coherence of LLM forecasts. The authors build a forecasting environment from historical stock returns and quantify incoherence by the profit obtainable from arbitraging a model's forecasts.
Key findings: forecasts become more incoherent as logical relations between events increase (joint vs. marginal distributions), and even innocuous added context like "I'm feeling optimistic" or "I had coffee this morning" degrades coherence.
Related event: Paper Tests LLM Probabilistic Coherence via Stock Arbitrage(2 posts)→
More from Research
- Anima Anandkumar's first podcast on Accelerated Understanding: 5T context and 4D physical AI — AnimaAnandkumar · 2026-09-05
- Uno pairs AR weights with diffusion LoRA adapters, beating EAGLE-3 and all diffusion LLMs — HongyiWang10 · 2026-09-05
- Self-explanation training generalizes beyond narrow hint formats to held-out evals — a_karvonen · 2026-09-05
- Two training targets from behavior investigations: counterfactual predictions and open-ended self-explanations — a_karvonen · 2026-09-05
- Anthropic Fellows train models to explain their own wild behaviors with generalization to held-out evals — a_karvonen · 2026-09-05
- Deep Learning Weekly #471: Claude Fable 5.1 launch, production-parity LLM evals, alignment paper — dl_weekly · 2026-09-05