LLM Forecasting Study: Activations More Reliable Than Text
burny_tech · x · 2026-07-11
Goodfire released a paper on LLM forecasting titled "What LLM Forecasters Know but Don’t Say".
The paper points out that while LLM forecasters often appear confident, their calibration is unstable, and their chain-of-thought may not reveal why predictions change. The research found that internal model activations reflect the true state better than the output text.
By using small probes to read these activations, the study aims to:
- Improve confidence calibration
- Detect "silent" shifts in evidence
- Partially recover prediction outcomes before reasoning begins
The authors argue this offers a cheaper and more faithful approach to auditing LLM predictions.
More from Research
- Speculative Diffusion Paper Maps the Speed-Quality Pareto Frontier via Two Tradeoffs — UCLA · 2026-10-26
- Git Records Fall Short for Auditing AI's Role in Research Repositories, Paper Finds — Koutian Wu · 2026-10-26
- What a 'Fused DeepLIFT' Actually Means: Gradient Overrides Without Hooks — jmschreiber91 · 2026-10-26
- New paper: AI agents negotiate badly under uncertainty and break deals on purpose — menhguin · 2026-10-26
- Cherimoya v0.3.1 Ships Fused DeepLIFT, ~7x Faster Attribution for Genomics Models — jmschreiber91 · 2026-10-26
- Proto-tools, a universal infrastructure layer for generative biology, now open source — jmschreiber91 · 2026-10-26