SlideOn's sparse dims are vocabulary-independent, enabling inverted-index retrieval
tomaarsen · x · 2026-10-20
In the SlideOn thread, tomaarsen explains the scoring mechanics: both models output 32,768-dim sparse vectors scored via dot product, but unlike many sparse modes the dimensions are independent of vocabulary tokens.
Active features can be stored in an inverted index: query features select posting lists, and matching weights contribute to document scores. The accompanying diagram walks through posting-list construction, candidate retrieval and weighted scoring for a "How is solar energy stored?" query, with irrelevant docs skipped entirely.
Related event: LightOn Open-Sources SlideOn Sparse Retrievers at 174M/339M, Claiming SOTA(23 posts)→
More from Research
- Mirror video planner shows emergent in-context learning, boosting robot lava-lake escape rate from 46% to 76% — vincesitzmann · 2026-10-20
- Training a small downscaler in 1 hour on one GPU cuts weather forecast error by up to 21% — yb2698 · 2026-10-20
- Progress-Aware Distillation Targets Stagnation in Small Language Model Agents — Yuanpu Cao · 2026-10-20
- HERA: co-evolving agent harnesses and environments for reliable abstention — UW · 2026-10-20
- HouseholdBench tests 13 LLMs on predicting household economic behavior; GBDT still wins — daveholtz · 2026-10-20
- Mirror author predicts an 'arm-farm' revival as video models make robot learning sample-efficient — vincesitzmann · 2026-10-20