Georgia Tech's EPSBench: LLM agents still trail analyst consensus on earnings forecasts
GeorgiaTech · hf · 2026-10-31
Georgia Tech researchers introduce EPSBench, a benchmark for evaluating LLM agents on quarterly EPS forecasting, covering 149 firm-quarters from 86 US companies.
- Setup: agents read filings, earnings-call transcripts and news, then estimate company-specific income-statement drivers with cited evidence; a shared accounting calculator converts these into GAAP diluted EPS for a common basis.
- Evaluation: six models under one harness, scored on point-forecast error vs date-matched analyst consensus and on whether prediction intervals hit stated coverage.
- Findings: three models show detectable paired differences in point error vs consensus, three have higher error; nominal 80% intervals contain reported EPS in only 37–70% of runs, with most misses above the upper bound due to systematically low forecasts.
More from Research
- After NVIDIA's BindClaw, every AI4Bio paper now needs an agentic pipeline demo to prove versatility — rishabh16_ · 2026-10-31
- Neural Thickets Revisited: Frozen-Model Logit Perturbations Match Full-Weight Perturbation Ensembles — kastnerkyle · 2026-10-31
- CS312 course post explains sharpness and the edge of stability in LLM pretraining — tatsu_hashimoto · 2026-10-31
- AI decodes wolf howls: 500K hours of Yellowstone audio analyzed by models — aza · 2026-10-31
- NBER paper: training your AI replacement can actually raise workers' pay — joshgans · 2026-10-31
- Trata's Hedge-Bench: no model tops 50% on real-world financial forecasting — ycombinator · 2026-10-31