PyroDash: a 4B model learns when to call a big LLM, cutting cost to 1/30
机器之心 · wechat · 2026-09-13
Pyromind's open-source PyroDash is a cost-aware, token-level small-large model collaborative inference paradigm: a small model (e.g. Qwen3.5-4B) generates first and emits a control token τoff when it can't continue; a Collaborate Engine packages the partial reasoning trace and hands it to a frozen large model (e.g. GLM-5.2) for one-way completion — at most one big-model call per request. Training has three stages: control-token embedding learning, SFT cold start, and cost-aware GRPO alignment where reward is accuracy minus λ times normalized cost (counting the small model's tokens twice — as its own output and as the big model's prefill input). On five math benchmarks, the cost-saving setting cuts cost from $49.36 to $1.78 (1/30) with only 3 points accuracy loss; the accuracy-focused setting reaches 64.04%, beating the pure large model's 57.68% at 20% lower cost, and outperforming RouteLLM and GlimpRouter baselines. Paper: arXiv:2607.20327; EasyHard-24k dataset open-sourced.
More from coding & agent
- astra is great at CAD itself, but telling it what to build remains the hard part — yacineMTB · 2026-09-13
- Yacine designs drone entirely from phone; custom NPU kernels max out chip without heatsink — yacineMTB · 2026-09-13
- GPT-Live-1 first impressions: most natural voice model yet, but instruction following is unreliable — kolchinski · 2026-09-13
- Open-source 'no-ai-slop' removes 20+ AI writing clichés, hits 8.7k stars — tom_doerr · 2026-09-13
- Devin SWE-2 first impressions: fast and cheap, but shorter endurance than Codex — CtrlAltDwayne · 2026-09-13
- Roboflow ships auto-label with GPT-6 Astra, making it the new default model — Roger_M_Taylor · 2026-09-13