What Are Sparse Embeddings: Vocab-Sized Vectors That Are 99% Zero
tomaarsen · x · 2026-09-17
In the SparseUp release thread, tomaarsen explains the basics: sparse embedding models embed text into vectors whose dimensionality often equals the vocabulary size, with 99% of positions at zero—only a few active dims.
This is an explanatory reply to Linkup's open-weight SparseUp release; details and benchmarks are in the main thread post.
Related event: Linkup Open-Sources SparseUp, Top Sub-150M Sparse Retrieval Model(6 posts)→
More from Research
- Rumor: OpenAI close to solving Hodge Conjecture, treading carefully with math community — Hesamation · 2026-09-17
- Study of 68,000 prompts finds users increasingly treat LLMs as oracles, often unaware — ValerioCapraro · 2026-09-17
- Liquid AI's small Longevity models beat GPT-5 and Claude Opus on Cell-published aging benchmarks — helloiamleonie · 2026-09-17
- Hanyang's 1,000-unit magnetic microrobot swarm lifts 350x unit weight — TinfoilTricorn · 2026-09-17
- Synthetic data boosts person detection mAP50 by 160%: AWS details industrial safety AI pipeline — AWS ML Blog · 2026-09-17
- Reka Labs lays out Omni-World Models: one architecture for language and physical AI — RekaAILabs · 2026-09-17