LegalBench author reflects on three years of the legal AI benchmark, plans LegalBench2
HazyResearch · x · 2026-11-07
Neel Guha reflects on LegalBench, the large-scale legal reasoning benchmark he led roughly three years ago. Core verdict: the thesis that LLMs would matter for legal AI — and that the field needed a benchmark to track it — proved right. DatologyAI/Thomson Reuters, Equall.ai and NVIDIA all used it; no single breakthrough came from it, but many small insights aggregated into real progress. He admits what went poorly and is building LegalBench2. A rare first-hand look at how domain benchmarks actually drive research.
Related event: LegalBench Author Reflects on Three Years of the Legal AI Benchmark(2 posts)→
More from Research
- Nearly 30% of ICLR 2027 reviews mention "appendix", fueling AI-written review fears — nsaphra · 2026-11-07
- LLMs Edit Robot Policy Weights Directly: 2,048 Parameter Tweak Lifts Success from 68% to 88% — zhaoran_wang · 2026-11-07
- GTEx MCP Server gives AI assistants 25 tools to query genomics across 54 tissue types — modelcontextprotocol · 2026-11-07
- Stanford's Anshul Kundaje: ML conference peer review is broken and unfixable — anshulkundaje · 2026-11-07
- Claude scanned 15 billion predicted proteins to crack the first 3D structure of vault particles — winer666 · 2026-11-07
- Veil 2.0 released: verify distributed protocols in Lean 4 with 2–20x faster model checking — tianyin_xu · 2026-11-07