Trata's Hedge-Bench: no model tops 50% on real-world financial forecasting
ycombinator · x · 2026-10-31
Trata released a revised Hedge-Bench benchmark linking open-ended reasoning to prediction ability: 92 real-world environments built from hedge fund analysts' reasoning traces, where agents forecast from frozen information snapshots and are scored against actual future outcomes. No model scores above 50%, with GPT-6 Astra leading at 44.2% — a wide gap between plausible-sounding reasoning and economically valuable forecasts in semi-verifiable domains like finance. The team also post-trained Meta's Muse-Glimmer-30B on 165 environments with per-turn credit assignment via LLM-as-a-Judge.
More from AGI Musings
- ML veteran Burkov: superintelligence abundance would drain Earth's remaining resources — burkov · 2026-10-31
- Santa Fe Institute's Eric Smith dismantles anthropocentrism in careful terms — danfaggella · 2026-10-31
- Bindu Reddy predicts 12 months of AI trends: tokens -80%, usage 10x, coding jobs shrink — bindureddy · 2026-10-31
- Accumulating wealth may help build a civilization where wealth itself becomes meaningless — Dr_Singularity · 2026-10-31
- Carl Benedikt Frey asks why some societies are far more suspicious of new technology — carlbfrey · 2026-10-31
- Thoughtworks CTO: Good developers wary of agentic coding because 'built' isn't 'working in production' — rseroter · 2026-10-31