0.1B SPLADE sparse retrieval beats Qwen3-0.6B dense embeddings in local RAG tests
jeffrey-0711 · reddit · 2026-11-06
A developer building a local RAG agent found that SPLADE-style sparse retrieval (semantic BM25) beats dense embedding models while using far less memory.
Using KURE-Sparse-nano (0.1B params, strong on Korean/English), he benchmarked 5,394 files / 52,851 chunks / 594 queries from his own Mac:
- SPLADE (sparse): MRR@10 0.67 at 0.5s median latency — best on both axes
- Qwen3-Embedding-0.6B dense: MRR@10 only 0.49, and nearly unusable on a 16GB MacBook Air (overheated and rebooted)
- SPLADE inference uses under 1.5GB VRAM, fits small machines
- Counterintuitively, hybrid SPLADE+dense retrieval (0.60) performs worse than SPLADE alone
The full implementation is open-sourced as AutoRAG Agent, with optimized KURE-sparse-nano inference and a Rust LanceDB inverted-index implementation.
More from coding & agent
- Frontier3DBench: 145-task benchmark tests coding agents on building interactive 3D worlds — _akhaliq · 2026-11-06
- Amazon research: self-distillation pushes weight memory to 59.9%, beating retrieval — omarsar0 · 2026-11-06
- Liquid AI's decision recipe matches Jev; new release adds on-device, modalities, primitives — helloiamleonie · 2026-11-06
- Decision models: general-purpose classifier or LLM? LoRA fine-tune wins — helloiamleonie · 2026-11-06
- LLMs are trained to predict next tokens, not make decisions — helloiamleonie · 2026-11-06
- Turn any LLM into a decision model with ~15 lines of inference-only code — helloiamleonie · 2026-11-06