NVIDIA and Cedana serve a portfolio of coding models on one 8x B200 node
josh_wills · x · 2026-10-02
NVIDIA and Cedana published a joint solution for hosting a full portfolio of frontier coding models on a single 8x B200 node.
- Architecture: NVIDIA Dynamo serves, NeMo Switchyard routes and escalates when a smaller model fails, and Cedana switches models in about a minute with sessions intact.
- Motivation: Enterprises self-host coding models for code privacy and cost control, but coding work spans autocomplete, small edits, complex debugging, and repo-wide planning — no single model is optimal across all.
- Key data: Per TraceLab (4,300 public Claude Code/Codex sessions), the median request finishes in 38 seconds, while 6% of requests over 10 minutes consume 69% of total request time — so a resident mid-size model serves most traffic, with the frontier model only for the tail.
- Portfolio: DeepSeek-V4-Pro-NVFP4 (873 GiB) for long-horizon planning, Kimi-K2.6 / GLM-5.26 for frontier tasks, plus smaller tiers for medium complexity.
Related event: Cedana and NVIDIA host full coding model fleet on one node(2 posts)→
More from coding & agent
- Generic API-catalog MCP server vs one tool per job: a token-efficiency tradeoff — MandeepKaur9407 · 2026-11-16
- sessionwiki turns your past coding-agent sessions into a searchable local wiki for agents — Cold_Past7184 · 2026-11-16
- Developer hits 700+ commits/day with AI, asks how to manage parallel agent work — blixt · 2026-11-16
- Alibaba's G-CAST: governed continual self-training keeps e-commerce agents fresh in production — Yuechun Yu · 2026-11-16
- harness-evals: open-source library to compare agent versions with success-vs-cost plots — slavadubrov · 2026-11-16
- Developers poll real costs of running coding agents on remote VMs for work — radim11 · 2026-11-16