Cutting AI inference bills from $250K to $7.6K/month: Eugene Cheah's tokenomics 101
AI Engineer · youtube · 2026-10-17
At AI Engineer NYC 2026, Eugene Cheah (Featherless AI) lays out four moves for teams spending at 2T tokens/day scale — with a cover case claiming a monthly bill cut from $250K to $7.6K (95%).
The four moves
- Measure: stop sharing one API key across teams; split keys to see real spend; understand agentic vs. process workload cost profiles.
- Right-size: classifiers as a new class of cheap models; "plan with the smart model, build with the cheap one"; give teams model labels, not model names; bootstrap fine-tunes and validate with shadow traffic.
- Buy capacity: 2x speed = 3x cost; bursty traffic vs. dedicated capacity tradeoffs.
- Check processes: tokens aren't work done; watch out for "slop grenades."
Q&A covers fine-tuning datasets, golden traces, escaping harness lock-in, and caching/auto-routing. A near-hour systematic tutorial on inference cost engineering, full timestamps included.
More from coding & agent
- Community ports DeepSelect to Hopper with CuTe DSL in TokenSpeed PR — zhyncs42 · 2026-10-17
- Computer-use agent consensus: escalate from weak to strong models instead of running frontier everywhere — xwang_lk · 2026-10-17
- Claude Code refuses tasks directly—but happily delegates them to Codex — AIFlow_ML · 2026-10-17
- 99% of the code written by Opus: solo dev ships a 20MB multiplayer mecha game — m0dE · 2026-10-17
- shadcn begs for early access to Anthropic's internal Claude Code model: 'progress has been ridiculous' — shadcn · 2026-10-17
- Perplexity reveals pplx-decider: reranker with zero reranking supervision, private MTEB-style eval — bo_wangbo · 2026-10-17