Criticizing dot product retrieval, advocating for inference scaling
lateinteraction · x · 2026-08-26
The author argues that in ML, retrieval tasks still rely on inert scoring functions (like the simple dot product), forcing the use of exponentially larger embeddings for basic improvements. The author advocates for "inference scaling for retrieval," also known as Late Interaction from 2019/2020. Following this, the author hinted at taking matters into their own hands to implement these ideas due to the inertia in the field.
Related event: 307M Late Interaction Model Beats 26x Larger Qwen, Sparking Storage Debate(4 posts)→
More from Research
- Goodfire's Nick Cammarata: neural networks are "already clean" in a possible interpretability breakthrough — tszzl · 2026-10-15
- tszzl: interpretability breakthrough feels like my understanding of neural nets jumped five years — tszzl · 2026-10-15
- LightSuite: fast cell detection and whole-brain atlas registration on a standard workstation — hisspikeness · 2026-10-15
- Pathway CSO pitches BDH as post-Transformer architecture uniting memory, reasoning, adaptation — schwentker · 2026-10-15
- AI-designed molecular scissors cut Alzheimer's proteins — but is it really AI? — gerardsans · 2026-10-15
- EmbodiedEvo tops RoboDojo at 38.95% success by letting robots inherit failed practice — thetripathi58 · 2026-10-15