RLI Eval: AI Automation Rate Jumps from 2.5% to 16.1% in a Year
sebkrier · x · 2026-07-05
An evaluation tracking AI automation capabilities (RLI) shows the best automation rate has risen from 2.5% to 16.1% since launch. However, AI still falls short of professional quality on most projects—none of the three tested Fable 5 deliverables could be accepted as final products. Despite this, the automation rate has improved rapidly in less than a year.
More from coding & agent
- Hugging Face wraps 6-month Training Agents series with open recordings and code — SergioPaniego · 2026-10-26
- Agent verified 'success' after double-voting: a lesson in harness vs model bugs — TejasKumar_ · 2026-10-26
- diginomica: agentic automation hype is overdone, and 'The Hackening' of enterprise security has begun — jonerp · 2026-10-26
- 'Vibe verification': ~60% of AI-written formal specs prove the wrong property — basedjensen · 2026-10-26
- Devin launches Security suite powered by Claude Mythos with $100M+ credits — DevinAI · 2026-10-26
- HackBench injects reward hacks into agent trajectories to stress-test monitors — DanielKhashabi · 2026-10-26