Bending Spoons runs 99% of AI traffic on self-hosted open models, thanks to evals
alex_verem · x · 2026-09-25
Bending Spoons runs 99% of its AI traffic on self-hosted open models. The author notes most CTOs default to frontier models like Claude Opus for agents but can't prove a cheaper model would work. The fix: build an eval set of production cases with expert-signed correct answers, run every candidate model against it, then treat the model as a shoppable line item — cheap models for routine work, frontier models only for hard cases.
More from coding & agent
- Runway MCP Lets Any AI Agent Generate Video Directly — tlakomy · 2026-09-25
- Hexis: open-source Postgres-native cognitive architecture giving LLMs memory and identity — GeretsOliver · 2026-09-25
- AgentArk, distilling multi-agent debate into a single LLM, accepted to NeurIPS 2026 — jindong_wang92 · 2026-09-25
- Ben Tossell Built an 87-Device Retro Timeline Site in a Morning with Codex and Subagents — Ben's Bites · 2026-09-25
- video-shotcraft author: Opus 5.5 makes a full code-generated video with music in one prompt — caolonghua · 2026-09-25
- Stability Matrix hang on Linux traced to 143 Windows fonts breaking Fontconfig — Signal_Care6558 · 2026-09-25