Kimi K3 Leads in Autonomous Legal Work Benchmark
rohanpaul_ai · x · 2026-07-19
In a rigorous benchmark test for autonomous legal work, Kimi K3 took a massive lead with a 26.7% success rate, far outpacing the runner-up Claude Fable 5 (14.2%).
The test comprises 120 real-world tasks across 24 legal domains (such as drafting memos and discovery summaries), requiring models to autonomously read case files and generate complete legal documents. Grading is exceptionally strict—missing any single detail results in a failed task, explaining why even the leader's success rate sits at only 26.7%.
Overall, completing only about 27 out of 100 tasks indicates that AI still has a long way to go in fully unsupervised legal work.
Related event: Kimi K3 Leads Harvey Legal Benchmark(4 posts)→
More from Models
- KOL defends B-tier rating for GPT-6.1 Sol, blames OpenAI's botched release naming — Angaisb_ · 2026-10-27
- php2shell exploit benchmark: only 2 of 5 frontier models can port the bug — xeophon · 2026-10-27
- Qwen Code v0.26.2-preview.0 adds session permission rules and experimental shared agent loop — qwen-code-review-bot · 2026-10-27
- GPT-6.1 Terra cuts factual error rate from 22.3% to 4.3%, an ~81% drop — haider1 · 2026-10-27
- GPT-6.1 Terra lands: early users praise speed, value and subagent performance — BenBajarin · 2026-10-27
- Leaked GPT-6.1 Terra benchmarks: cheap but 'boring', still far behind Astra and Claude — Angaisb_ · 2026-10-27