Depth Memory Framework Unifies Benefits of Deeper Layer Mixing

A new work proposes Depth Memory, a unified framework viewing each layer as reading from and writing to a set of streams, explaining why richer depth-wise mixing (as in mHC and AttnRes) improves performance. The approach shows strong results on DepthBench.

2026-10-15 ~ 2026-10-15 · 2 related posts

• Depth Memory framework unifies mHC and AttnRes, equipping layers with matrix-valued memory — orvieto_antonio · 2026-10-15
• Linear Depth Memory: matrix-valued memory tops PreLN and mHC on DepthBench across scales — SonglinYang4 · 2026-10-15