New GDPx Benchmark: Frontier Models Score Near 0% on Full Legal Jobs
jkubicki · x · 2026-11-04
jkubicki cites the new GDPx benchmark, which had a single AI agent complete entire expert jobs across 11 fields (some up to 8 hours); every frontier model scored near 0% on legal tasks. But he argues this tests fully autonomous end-to-end work, not how successful lawyers actually use AI—as collaborators who decompose tasks, check outputs, and stay accountable. The takeaway: AI can't yet do legal work alone, making lawyers who can steer agents more valuable, not less.
More from AGI Musings
- Inside the AI data grey market: token resellers make millions until a lab buys back its own data — vista8 · 2026-11-04
- AI policy after the election: legislative reset, hearings, and a likely market correction — ShakeelHashim · 2026-11-04
- Greg Kamradt: benchmarks should measure AI's contribution to the frontier of knowledge — GregKamradt · 2026-11-04
- BioSimBench debuts: 27B RL-trained model beats frontier LLMs at simulating cell biology — omarsar0 · 2026-11-04
- Many 'creative' jobs are really 'communication' jobs, argues viral take — floguo · 2026-11-04
- Sam Altman: a trillion 'modest AIs' beats one AI with a trillion IQ — KyeGomezB · 2026-11-04