LegalForecastBench evals AI on predicting real judicial outcomes
BlackHC · x · 2026-10-14
- Building good legal AI benchmarks is getting hard: frontier models ace simple, objectively verifiable legal tasks, and what's left mostly involves judgment with no clear ground truth.
- LegalForecastBench targets a high-level legal task with objective ground truth: predicting how judges will rule on real motions.
- Using real-world judicial outcomes enables quantitative evaluation of deep legal reasoning, sidestepping the discretion problem that plagues legal evals.
More from Research
- Google Fellow John Platt: AI rerouting cut flight contrails by 62% — ziv_ravid · 2026-10-14
- Bible verse continuation triggers malevolent outputs, more evidence of subliminal learning — sebkrier · 2026-10-14
- ReRoute: physics-informed fine-tuning cuts counterfactual error up to 89.6% — AnimaAnandkumar · 2026-10-14
- Spork post-training recipe lifts chess puzzle solve rate 100% over best GRPO setup — aidenybai · 2026-10-14
- Dyna-2 Deep Dive: World-Action Models Trained on 1M Hours of Human Video, 99.4% 24-Hour Autonomy — Latent Space · 2026-10-14
- Procedural Core: a 1M-param abstract model boosts 85M-param ViT ImageNet accuracy by 2.2 points — kastnerkyle · 2026-10-14