NVIDIA's ELR removes the one-loop tax: Koi-4B scores 18 at one loop, 29 at four loops

nvidia · hf · 2026-10-12

NVIDIA presents Elastic Latent Recurrence (ELR), a recipe that eliminates the one-loop tax of latent recurrent language models, where one-loop performance normally falls well short of same-size non-recurrent models.

At one loop, an ELR model costs the same to serve as a non-recurrent model of the same size and matches its accuracy; each additional loop buys more accuracy for more compute.
Three key choices: a gated state update with learned step size, loop counts mixed within each optimizer update, and conditioning each pass on its loop index and budget.
Mixed depths can be introduced late in pretraining — with zero-initialized conditioning this even beats mixed-depth training from scratch; in MoE models, reusing first-loop expert routes cuts mixed-depth update time by 11.7%; inference keeps only the last loop's attention KV, holding memory at one-loop level.
On 4B dense and 30B-A3B MoE hybrids, one-loop results are within 0.52 / +1.04 points of baselines on a ten-task average; four loops add 7.25 / 4.75 points. Post-training yields Koi-4B: an estimated 18 on the Artificial Analysis Intelligence Index at one loop (matching Qwen3.5-4B) and 29 at four loops.

One checkpoint now flexibly trades latency for accuracy.

Original post →

NVIDIA's ELR removes the one-loop tax: Koi-4B scores 18 at one loop, 29 at four loops

More from Infra