dair-ai founder: own your evals — public benchmarks won't tell you what to ship
dair_ai · x · 2026-11-09
dair-ai founder Omar Sansevero argues that while public benchmarks remain useful for research and discussion, the only way to trust model choices in production is to build your own evals. He frames evals as increasingly central to unlocking new use cases and eventually RSI-driven business optimization — something companies should own in-house rather than rent, and dair-ai has started helping companies build this capability internally. The post quotes a discussion comparing today's eval industry to a military-industrial complex.
More from coding & agent
- CS spent 70 years killing nondeterminism; in 2026 we set temperature=0.8 — korymath · 2026-11-09
- Open-source Bello claims 3x more usage from your coding agent at same quality — nikola_mr64990 · 2026-11-09
- Nominal raises $240M Series C at $3B valuation for AI hardware engineering — graceisford · 2026-11-09
- HF's Niels Rogge releases 'Getting Started with Local AI' slides: quantization, MoE, local agents — NielsRogge · 2026-11-09
- A 24/7 pixel-art TV channel for an invented sport, built entirely by Claude Code — spite_as_an_arrow · 2026-11-09
- GPT-4o silently drops schema invariants at turn 18 in 50-turn agent stress test, token burn hits 24.2x — ParrotIntegrated · 2026-11-09