NVIDIA's Elastic Latent Recurrence lets one 4B model trade loop count for accuracy: 18→29 on AA Index

burny_tech · x · 2026-11-02

NVIDIA published Elastic Latent Recurrence (ELR), a training recipe that removes the "one-loop tax" of looped language models, which normally underperform same-size standard transformers when run once.

Recipe: gated state updates with learned step size, loop counts mixed within each optimizer update, and per-pass conditioning on loop index and budget. Mixed depths can be introduced late in pretraining with zero-initialized conditioning, reaching lower loss than training mixed from scratch.
Results: a 4B ELR model is within 0.52 points of its standard baseline at one loop; four loops add 7.25 points on a ten-task average. In MoE models, reusing first-loop expert routes cuts mixed-depth update time by 11.7%; keeping only the last loop's attention KV cache holds memory at one-loop levels.
Deployment: post-training yields Koi-4B, a single checkpoint whose estimated Artificial Analysis Intelligence Index goes from 18 (matching Qwen3.5-4B) to 29 just by increasing loop count.
The upshot: one checkpoint can trade inference compute for accuracy at deploy time.

Original post →

NVIDIA's Elastic Latent Recurrence lets one 4B model trade loop count for accuracy: 18→29 on AA Index

More from Models