Rippling Benchmark: Opus Edges Out, $7B OpenRouter Deal Hints at Smart Routing Value
serendip-ml · reddit · 2026-08-19
Rippling ran a benchmark testing 15 models on real payroll work with 2,100 scored runs each. Results showed untuned models scored 88.5%-89.5%, Z.ai's GLM 5.2 hit 88.7% for $621, and Anthropic's Opus 4.6 (the only prompt-tuned model) reached 91.0% for $1,453, yet still failed 9% of runs. Weeks later, a hypothetical $7B acquisition of OpenRouter by Stripe is noted. OpenRouter processes 100 trillion tokens monthly for 8M developers, earning $140M ARR. The key question raised: How much agent inference requires heavy thinking versus simple structured output? Is smart routing essential, or is a fixed role-based setup sufficient?
More from coding & agent
- Gemini Image Generation Silently Fails From Hetzner IPs — Network Origin Was the Culprit — dota2dinall · 2026-08-19
- Vercel open sources fx: a tiny, fast native coding agent — Rasmic · 2026-08-19
- GitHub Repo Open-Sources Author Style Mimicry Prompts Featuring Ottessa Moshfegh — TuhinChakr · 2026-08-19
- Open source file upload service Byteship built with Grok released — jasonkneen · 2026-08-19
- ClawGym II paper: Improving agents via mixed-harness training — omarsar0 · 2026-08-19
- MacStories' Codex Automation Guide: Process Notes, Save Emails, Auto-Tag Read-Later — Dimillian · 2026-08-19