Running Qwen3.8 27B on a 2018 Jetson AGX Xavier: 106 tok/s prefill, power-brick fixes and all
cortexist · reddit · 2026-11-14
The author revived a dusty Jetson AGX Xavier (2018, 512-core Volta, 32GB LPDDR4x, 136 GB/s) as a local LLM box and documented everything.
Results (512-token prompt): after forking MLC-LLM and writing the Nemotron backend from scratch, Nemotron 3.5 Bolt E4B hits 880 tok/s prefill and 50 tok/s code decode; Gemma 4 12B does 215/35 tok/s; Qwen3.8 27B (DavidAU's TURBO-Fable-Cold-Fusion checkpoint) peaks at 106 tok/s prefill, holds 104 at 1K, and 13.4 tok/s decode with a Qwen3.8 2B draft.
Gotchas: the stock power brick browns out under sustained 35W load; a USB-C PD supply plus a short OWC USB4 cable kept it up for over a week. An untricked idea: cable two kits over PCIe 4.0 x8 (16 GB/s each way) and split model layers across them. Buying advice: 16GB modules run $130 on eBay (runs Gemma 4 12B), 32GB $300; get one with a carrier or budget $60 extra.
More from coding & agent
- HQ pitches an open-source 'company brain' giving team AI a shared context layer — jacob_posel · 2026-11-14
- Cohere Labs talk: Harness² recursively improves agent harnesses at test time — Cohere · 2026-11-14
- Someone Built a Provably Fair Dice Casino Where AI Agents Place Bets via MCP — adamamcbride · 2026-11-14
- Intern shipped a multiplayer app built with Claude Code, got hired, then presented it on stage — Alec_Coughlin · 2026-11-14
- Google Research showcases Antigravity's /teamwork for science & engineering — Google Research · 2026-11-14
- Dev: Working at a High Level with AI Is Great but Exhausting — jonathan_wilke · 2026-11-14