USC Introduces WhiteMatter: Cuts Cache Memory in Half, Outperforms Baseline with More Layers
burkov · x · 2026-08-21
Researchers from USC introduced WhiteMatter, a new Transformer architecture. By dynamically mixing hidden states across all layer depths into compact key-value channels, the model enables outperforming standard baselines with 50% more layers while halving cache memory consumption.
More from Research
- Discussion: Why Model Artifacts Look Like MSE Blur, Not Natural Ugliness — kalomaze · 2026-08-21
- Depth Anything V4 Uses Riemannian Flow Matching for Dynamic 4D Scene Reconstruction — RexDouglass · 2026-08-21
- View: LLMs Should Be Trained at Sampling Temperature; RL Targets Canonical Temp — kalomaze · 2026-08-21
- The academic feud between Judea Pearl and Guido Imbens on causal graphs — robseamans · 2026-08-21
- Tsinghua study introduces "Representational Equality" metric for LLM cultural bias — zibuyu9 · 2026-08-21
- AutoDesign: Self-Evolving Harness for Long-Horizon Agentic Design — 机器之心 · 2026-08-21