Red Hat's internal pilot: open-weight models cut token costs 40-70% on 1T tokens
AccBalanced · x · 2026-11-18
Red Hat ran its own engineering org as "customer zero" in the EnMaaS pilot (Praxis gateway + llm-d + vLLM) to give developers frictionless access to open-weight models:
- Scale: over 1 trillion tokens processed in a single 30-day window.
- Adoption flip: usage started 70/30 in favor of commercial frontier APIs and flipped to 30/70 toward internally hosted open weights, gaining share daily.
- Routing: Praxis intelligently routed most routine coding and evaluation traffic away from external APIs, which now handle only specialized workloads.
- Cost: shifting that traffic to self-hosted models on llm-d cut effective token spend 40-70% vs prior frontier pricing, depending on prompt caching and routing.
One of the most detailed first-hand datasets yet on enterprises replacing frontier APIs with open weights.
More from coding & agent
- Plannotator: 13K-star repo adds PR-style review to coding agent plans — techNmak · 2026-11-18
- Early access to Devin bots and a chat with one of their creators, Walden Yan — Madisonkanna · 2026-11-18
- Open-sourced genetic algorithm optimizer hit top 2 in Open Kernel Challenge within 7 hours — morgymcg · 2026-11-18
- Future codebases will be 80% AI-assisted, raising sloppier-code concerns — 996roma · 2026-11-18
- Two weeks of service_tier "priority" A/B tests: median +5.5% TTFT, basically noise — iambubu0523 · 2026-11-18
- Sentry founder jokes about MCP as HN thread declares it the standard for AI agent tooling — zeeg · 2026-11-18