2788 tokens/s on one RTX 5090: open-source ORL accelerates diffusion LLMs
arty_photography · reddit · 2026-10-17
A fully open-source research release, ORL (On-Policy Residual Learning) plus a custom inference engine dlmrt, dramatically speeds up diffusion LLMs: 2,788 tokens/s on a single RTX 5090 (LLaDA-2.1-mini, NVFP4), 7,708 tokens/s on 8×B200, and up to 6.9× end-to-end throughput on QwenDLM-4B while raising HumanEval pass@1 from 82.9% to 89.6%. ORL trains on fresh on-policy residual trajectories and uses Adjacent Entropy Contraction to finalize more tokens per denoising step. dlmrt captures the whole denoising loop in a CUDA graph and beats SGLang/vLLM by 1.2-1.6×. Models, training code, eval pipeline and engine are all open-sourced.
Related event: ORL Speeds Up Diffusion LMs to 2788 Tokens/s on a Single RTX 5090(3 posts)→
More from Infra
- vLLM makers Inferact partner with AMD to optimize agentic inference on Instinct GPUs — woosuk_k · 2026-10-17
- Zuck bets most inference runs locally: Meta emerging as the champion of frontier local AI — StefanoGogioso · 2026-10-17
- ModelSphere launches as open-source LLM serving platform for mixed GPU fleets — Codeblix_Ltd · 2026-10-17
- Flux 3 Klein benchmark: nvfp4 hits 13.7s per image vs 25.2s for fp8 — dh7net · 2026-10-17
- .NET 10 Lambda ARM64 base image ships with 170MB of leftover Yum cache — DanielLockyer · 2026-10-17
- Vera Rubin called a generational upgrade: fewer than 10% of US data centers ready within 12 months — sudoraohacker · 2026-10-17