Cloudflare's AI Bets: Small-Model Wager Missed, Edge Compute Wins as Latency Returns
michellechen · x · 2026-11-13
At the AI Engineer Code Summit, Cloudflare's Michelle Chen shared lessons from the company's AI bets: their 2023 wager that models would shrink was wrong—7B models grew into trillion-parameter ones, so they pivoted to serving giant LLMs. Now small fast System 1 models bring network latency back into play, giving edge compute an edge again. "We weren't wrong, we were just early."
Her slide shows a code-review agent running on a Durable Object: a "System One first, LLM last" pipeline picks tools with small models, makes 9 parallel tool calls, and totals 6 LLM calls / 39,374 tokens in 8.1s at $0.0064—correctly flagging a real jitter bug where backoff exceeded the 30s cap.
More from coding & agent
- AI's 3D game development leap: use Blender first, then integrate, dev recommends — chongdashu · 2026-11-13
- Last talk at DeepMind's Code Summit was the only one showing live source code — lucasmeijer · 2026-11-13
- DavidKPiano: Coding agent effort levels will disappear like planning mode did — DavidKPiano · 2026-11-13
- Hamel Husain Praises Top Context-Layer Company's Evals as Best Public Example — verrsane · 2026-11-13
- Developer: agents now write my issues, emails and chats — what am I even for? — m4rkmc · 2026-11-13
- Qwen Code Desktop v0.27.3 ships Skills import, agent retry fixes across 17 contributors — github-actions[bot] · 2026-11-13