Meta paper says RL can optimize code speed, with Qwen 2.5 7B and CWM 32B gains
burny_tech · x · 2026-07-29
Meta paper shows how to make reinforcement learning optimize code speed
This arXiv paper from Meta FAIR looks at a hard problem: once you add execution time to the reward, RL for code optimization becomes brittle because timing noise, sparse rewards, and GRPO instability overwhelm the signal.
What they changed
- Built DMC-Optim with 2,723 cleaned problems, plus large correctness and optimization test sets.
- Reworked the reward design so correctness and speed are combined more carefully.
- Used an offline simulator to choose promising configurations before online training.
- Adapted GRPO and evaluation for the noisier timed-execution setting.
Reported results
- On Qwen 2.5 7B, strict top-50% pass@1 rises from 18.0% to 31.3%.
- On CWM 32B, it rises from 30.7% to 50.4%.
- Under degraded timing sandboxes, robust optimization RL reaches 100%–200% better performance than standard RLVR depending on the metric.
- On LCB, CWM 32B wins up to 83% of median-sample speed comparisons against standard RLVR.
The core takeaway is that optimizing for speed is possible, but only if the test harness, reward shaping, and optimizer are all engineered for noisy execution-time signals.
More from Models
- Claude Opus 5 tops a cybersecurity benchmark but becomes noisier when it overworks — Thom_Wolf · 2026-07-29
- Anthropic is said to have removed reasoning traces from its UI surfaces — RileyRalmuto · 2026-07-29
- Pangram Raises New Round, Launches Stronger AI Text and Image Detection Models — deedydas · 2026-07-29
- Debunked: Viral Seedance 2.5 Videos Are Actually Old 2.0 Footage — toolstelegraph · 2026-07-29
- Olmo 3 talk covers DPO, data problems, and how research reaches frontier models — natolambert · 2026-07-29
- Google DeepMind opens staff scientist role for Gemini long-context and memory — LucaAmb · 2026-07-29