Chorus: Shared N-gram Embedding Tables Cut Table Size 87% While Beating Per-Layer Designs
Stephen Chung · hf · 2026-11-19
Stephen Chung and Wenyu Du of DualverseAI propose Chorus, a new take on n-gram embedding tables. Prior designs give each layer its own table — costly in memory and blocking knowledge sharing — while naively sharing one table starves layers of learnable preferences and underperforms.
Chorus shares a single set of n-gram tables across the whole network; each layer rescales embedding components by factors computed from its current hidden state and reads them through its own projection.
Results: with 1.2B table parameters, Chorus predicts text better than FactorEmbed and DeepSeek's Engram do with 9.6B — an 87% smaller table — and outperforms Engram, TN-gram and FactorEngram across table sizes from 20M to 9.6B. Paired with the memory-efficient optimizer Ember, it saves 99.2% of table-optimizer memory vs AdamW without degrading training.
More from Infra
- US blocks Chinese firms from renting Nvidia chips abroad while Microsoft nears $500M OpenAI deal for ByteDance — kevinnbass · 2026-11-19
- vikhyatk: we are overinvesting in GPU compilers, underinvesting in CPU compilers — mohbibi_ · 2026-11-19
- KVARK Flash: a proxy that caps DeepSeek's thinking per request, 30-50% fewer tokens — Ill_Contribution_717 · 2026-11-19
- Polymarket launches Anthropic IPO perps, ANT trades near $104 ahead of Nasdaq debut — Polymarket · 2026-11-19
- R9700 at 160W for llama.cpp: ~20% pp loss, near-silent fans — LippyBumblebutt · 2026-11-19
- Baseten runs DeepSeek v4.1 Flash at ~2,500 tok/s on Vera Rubin, 1.7x faster than GB300 — baseten · 2026-11-19