Measuring AI Performance with Custom Evals like KateBench
every · x · 2026-08-27
This article discusses how to measure the ROI of hefty token budgets using evals. It describes the development of KateBench, an AI copyeditor trained on 30,000 edits, and how evals are used to measure its performance on every piece of writing.
Related event: Every Trains AI Copy-Editing Tool KateBench on Editor's 30,000 Edits(2 posts)→
More from Companies & People
- Amazon blocks Grok bots; Musk jokes about starting an Amazon competitor — NicoVerderosa · 2026-10-15
- ROBIB robotics meetup #3 lands in Ghent on Nov 19 with academic and industry speakers — MarwaEldiwiny · 2026-10-15
- Bank of America puts 4,000 people on fixing vulnerabilities uncovered by AI models — The Decoder · 2026-10-15
- G2's 2026 AI Agents report: Distrust Tax, Shadow Agents and enterprise adoption — sanderssays · 2026-10-15
- Anthropic is hiring a DevRel for scientists to grow its Claude research community — AI4Code · 2026-10-15
- Lexsi's AI Safety & Alignment Hackathon offers $30K+ prizes with Cohere Labs and Hugging Face — Cohere_Labs · 2026-10-15