Chorus paper: shared n-gram tables cut memory 87% while beating DeepSeek's Engram

brown2green · reddit · 2026-11-20

A new arXiv paper proposes Chorus, a shared n-gram embedding design that gives language models extra capacity at minimal compute cost.

The poster notes Engram tables can scale well beyond backbone size with performance tracking table size, dispelling the idea that only 25% of weights can be Engram layers.

Related event: Chorus: Shared n-gram Embedding Table Shrinks 87% While Quality Rises(2 posts)→

Original post →

More from Infra

Infra channel →