DUET (NeurIPS 2026) splits RL token budgets to beat GRPO with 2.51x speedup
xuandongzhao · x · 2026-10-02
DUET (DUal-controlled tokEn allocaTion), accepted to NeurIPS 2026, is a joint allocator that splits a fixed training-time token budget across prompts (rollout count) and per-rollout max-token caps under a single Lagrange multiplier. Training Qwen3-1.7B-Base on MATH: DUET at half the rollout budget matches or beats full-budget GRPO across five reasoning benchmarks (MATH-500, GSM8K, AIME-2024, HumanEval, GPQA-Diamond) with a 2.51x wall-clock speedup, dominating the budget-accuracy frontier at every budget point. Upgraded to vLLM V1 (0.16).
More from Infra
- LithosAI launches LithosBox: millisecond snapshot-and-fork sandboxes for AI agents — JiaZhihao · 2026-10-03
- Wish list: a Qwen4 27B with 100B+ Engram offloaded to RAM and NVMe for local users — casper_hansen_ · 2026-10-03
- State of Local AI 2026: one gaming GPU now matches the world's best model from Feb 2026 — Scobleizer · 2026-10-03
- Leak: DeepSeek V4.1 Pro ~2T params, trained sparse from scratch with no instabilities — teortaxesTex · 2026-10-03
- Stas Bekman demos his cluster-audit AI skill running on an AWS cluster — StasBekman · 2026-10-03
- Running 180B Qwen3.8-Flash-Next at 40-50 t/s on 64GB RAM + 16GB VRAM with Strata — danamir_ · 2026-10-03