Universal YOCO paper combines recursive compute with efficient attention for depth scaling
donglixp · x · 2026-09-11
A tweet chain points to the arXiv paper 'Universal YOCO for Efficient Depth Scaling' (Yutao Sun, Li Dong, Furu Wei et al.). YOCO-U merges the YOCO decoder-decoder architecture with recursive computation: a parameter-shared Universal Self-Decoder iterates multiple times, confined to shallow efficient-attention layers, giving a constant global KV cache, linear pre-filling, and deeper representations at limited overhead. It stays competitive on general and long-context benchmarks, suggesting efficient-attention + recursion as a promising scaling direction.
Related event: Microsoft's Universal YOCO enables efficient depth scaling(3 posts)→
More from Research
- Paper: data schedules and learning rate schedules should be designed together — alifcoder · 2026-10-27
- Open Models Match GPT-6 on MedMCQA but Trail 17 Points on Hard Clinical Cases — keunwoochoi · 2026-10-27
- Data Curriculum and LR Schedule Should Be Co-Designed: Math Data Placement Flips With Learning Rate — CatAstro_Piyush · 2026-10-27
- Health-Focused Benchmark Reranks LLMs: Muse Spark Open Jumps from #9 to #1 — keunwoochoi · 2026-10-27
- Open-source Latch compiles PyTorch into training megakernels, 2.75x faster on H100 — sguada · 2026-10-27
- Goodfire commits $10M to a 'Human Genome Project for AI' to reverse-engineer frontier models — CatAstro_Piyush · 2026-10-27