Crosby's 27B model beats frontier LLMs at judging surgical redlines, 90.6% vs 84.3%
graceisford · x · 2026-11-04
Crosby trained a 27B model, with its attorneys defining what makes an edit 'surgical,' to judge contract redlines. It hits 90.6% accuracy vs 84.3% for the next-best frontier model, at a fraction of the cost ($0.13/M tokens per the score-vs-spend chart). The thesis: messy AI redlines (full-paragraph rewrites, edit spam) erode client trust, so defining 'great' yourself and training on it is the future of evals — steering both agents and evaluation. Detailed in the post 'How to Train your Surgeon'.
Related event: Crosby Launches Surgeon, a Model for Judging Surgical Contract Edits(2 posts)→
More from Models
- '6 Astra High' spotted in model picker, hinting at new Astra release — gethackteam · 2026-11-04
- DeepSeek V4.1 Flash reads raw weights: 3 of 4 RL checkpoints identified correctly — poppear · 2026-11-04
- Blogger: Everyone's Sleeping on Google's Latest Models — 'They're Back' — iruletheworldmo · 2026-11-04
- Blogger Claims OpenAI Astra 6.2 Cuts Price 60% to Match Opus 5.5 — bindureddy · 2026-11-04
- Sarvam Prism launch demoed with a Snake game comparing decision models vs LLMs — itsOmSarraf_ · 2026-11-04
- Aohan: inference should be fast enough that the environment becomes the bottleneck — _arohan_ · 2026-11-04