MSRA team unveils Universal YOCO: shared-parameters recursion for efficient depth scaling
donglixp · x · 2026-09-11
Researchers at Microsoft Research Asia (Yutao Sun, Li Dong, Furu Wei, et al.) released Universal YOCO (YOCO-U), arXiv:2604.01220. The paper argues standard Transformers scale inference-time compute poorly: looping strategies carry high overhead and KV caches grow with depth. YOCO-U combines the YOCO decoder-decoder architecture with recursion — a Universal Self-Decoder iterates via parameter sharing, confined to shallow efficient-attention layers. YOCO contributes a constant global KV cache and linear prefill, while partial recursion adds representational depth at limited cost. Empirically YOCO-U stays competitive on general and long-context benchmarks. The authors frame it as architecture innovation's 'second half': moving from component innovation to layout innovation, with Looped YOCO completing the stack.
Related event: Microsoft's Universal YOCO enables efficient depth scaling(3 posts)→
More from Research
- AI folks, stop saying "biology" when you mean drug development — KevinKaichuang · 2026-10-26
- ostris Pretrains 'Primal VAE': Aggressive 2x-Downsample, 1-Channel Latent Design — ostrisai · 2026-10-26
- ostris pre-training "Primal VAE": 1-channel latents match modern 8x VAEs via patching — ostrisai · 2026-10-26
- Shanghai AI Lab-led team releases 100-hour egocentric wearable dataset for humanoid training — ChuckDBrooks · 2026-10-26
- ConvFill: A 0.6B Talker Buys Time While a Large Reasoner Whispers the Real Answer — andimarafioti · 2026-10-26
- Flow Reasoning Models turn flow matching into iterative reasoners, hitting 99.5% on Sudoku-Extreme — alec_helbling · 2026-10-26