Three small MoEs tested in one agent pipeline: Flint 40B fastest and cheapest, Nemotron chokes on edits
GapNew4766 · reddit · 2026-10-23
The author benchmarked three small MoEs released within weeks — Flint 40B-A2.5B, Ornith 1.5 35B-A3B, and Nemotron 3.5 Lightning 30B-A3B — in an identical local agent pipeline: Opus 5.5 plans and reviews via OpenRouter while three parallel local workers on RTX PRO 6000 Blackwell 96GB (Q80 GGUF, llama.cpp) build a 20s Three.js chain-reaction scene from one prompt (1,000–1,700 lines, 6–9 files).
Key findings
- Flint was most efficient overall: 7:36 wall time, 6 fix rounds, $0.41 Opus spend. Ornith: 9:04, 14 rounds, $0.49. Nemotron had the best raw decode/prefill but spent 17 extra minutes in fix waves.
- Raw speed didn't decide the run: Nemotron generated 5x more tokens, made 2x more requests, and its edit success rate was 17% (23 ok vs 109 failed), repeatedly piling stray braces into the same file.
- The worker sets the planner's bill: same Opus, same prompt, spend swung $0.41→$0.95 by swapping the underlying model. Opus I/O ratio was 48–76x — you pay for reading worker output, not writing code.
- Log oddities: one Ornith worker overwrote its 367-line file with a literal ... and reported ok; a Flint worker read the same three files 23 times without writing.
Caveats: one run per model (no variance measured), CUDA support landed a day earlier, output quality not yet evaluated, none passed first try. The author works on Atomic Agent, the open-source agent used.
More from coding & agent
- NVIDIA unveils OpenShell Reference Architecture to set boundaries for enterprise AI agents — NicoVerderosa · 2026-10-23
- Most 'unauthorized' AI agents in US bank accounts were let in by customers themselves, per Axios — nptacek · 2026-10-23
- Personal AI agents become a capital-intensive race as cloud execution environments reshape cost structures — sujingshen · 2026-10-23
- LLMs make code so cheap that recursive rewrites could make verified software tractable — jd_pressman · 2026-10-23
- Hugging Face hands OpenEnv RL environments project to Linux Foundation for neutral governance — lmoroney · 2026-10-23
- Theo writes the definitive explainer on what an AI coding "harness" actually is — pvncher · 2026-10-23