LoopUno: 52-Minute LoRA Training Makes 3B LLM Draft 2 Tokens per Step Losslessly

LucaAmb · x · 2026-10-26

Researchers from UC Riverside and Georgia Tech introduce LoopUno, a diffusion-style drafting method that accelerates looped Transformer LLMs — arguing diffusion is a sampling method that should sit on top of autoregressive (AR) models rather than replace them.

How it works

No separate drafter: the base Nanbeige4.2-3B (22 layers × 2 loops) stays frozen; one gated LoRA activates only on noise positions, drafting a block of 8 tokens in a single forward pass. The unmodified model verifies all guesses in one forward and keeps the longest correct prefix, so output is lossless.
Self-distillation training: each batch contains a clean copy (teacher) and a block-noised copy (student); the loss is the total-variation distance between next-token distributions, which exactly equals 1 − acceptance rate. No labels or drafter corpus needed.
Training takes just 60 steps / 31M tokens / 52 minutes on 8×A100, reaching 2.04 tokens per step.

Results

1.47× end-to-end decode speedup (2274 vs 1545 tok/s on gsm8k); function calls 1.98× faster, math reasoning 1.47× faster.
Accuracy unchanged at 96.0% (±1.7% sampling noise across checkpoints).
After only 14 minutes (8M tokens), already 1.13× faster end to end.

The approach sidesteps classic speculative decoding pain points: drafter training time, corpus construction, and acceptance-rate collapse on out-of-domain data. Adapters are on Hugging Face; code is open source.

Original post →

LoopUno: 52-Minute LoRA Training Makes 3B LLM Draft 2 Tokens per Step Losslessly

More from Models