VAGOsolutions releases tiny 1.5M-115M MoE ColBERT retrieval models

VAGOsolutions has released the SauerkrautLM-MoEdernColBERT family on Hugging Face — three multilingual multi-vector retrieval models with just 1.5M, 27M, and 115M parameters, open-sourced under Apache 2.0. According to tomaarsen, these are ColBERT-style multi-vector retrieval models built on a ModernBERT hybrid MoE architecture; he noted he has never seen MoE embedding/retrieval models this small, with support for German, English, Spanish, Dutch, and other languages.

Confirmed

• The family comes in 1.5M, 27M, and 115M parameter sizes, open-sourced under Apache 2.0 and now available on Hugging Face
• Architecture is a multi-vector ColBERT retrieval model based on a ModernBERT hybrid MoE design
• Official comparison charts show the 115M version outperforming Qwen3-Embedding-0.6B on 9 of 11 benchmarks, with roughly 8x the throughput
• The 27M version scores 0.684 vs 0.681 Recall@20 on the cross-lingual MKQA benchmark (English Wikipedia retrieval from six-language queries), beating Qwen3-Embedding-0.6B with about 1/22 the parameters
• The 1.5M version narrowly edges out IBM granite-97m-multilingual-r2 on English NanoBEIR, 0.591 vs 0.589 nDCG@10

Why it matters

• Models this tiny can approach or even surpass big-vendor 0.6B-class embedding models, meaning retrieval workloads can achieve comparable quality at far smaller scales, sharply cutting deployment costs
• 8x the throughput of Qwen3-Embedding-0.6B is a real engineering advantage for high-concurrency retrieval services
• Bringing MoE architectures down to the 1.5M–115M range for embedding/retrieval models offers a rare open-source reference implementation for this direction

2026-10-26 ~ 2026-10-26 · 10 related posts

• [source] MoEdernColBERT-115M matches Qwen3-Embedding-0.6B at 8x throughput — tomaarsen · 2026-10-26
• [source] VAGOsolutions releases MoE retrievers as small as 1.5M parameters, Apache 2.0 — tomaarsen · 2026-10-26
• How to run MoEdernColBERT with Sentence Transformers MultiVectorEncoder — tomaarsen · 2026-10-26
• [source] 115M MoE ColBERT beats Qwen3-Embedding-0.6B on 9 of 10 quality benchmarks — tomaarsen · 2026-10-26
• 1.5M-parameter retriever edges out IBM's 97M granite on NanoBEIR — tomaarsen · 2026-10-26
• MoEdernColBERT-27M beats Qwen3-Embedding-0.6B with 22x fewer parameters — tomaarsen · 2026-10-26
• MoEdernColBERT-1.5M encodes 43,255 docs/s, 300x faster than Qwen3-0.6B — tomaarsen · 2026-10-26
• How MoEdernColBERT stays tiny: 2-of-8 experts plus hash embeddings — tomaarsen · 2026-10-26
• Why tiny encoders hold up: multi-vector retrieval trades storage for quality — tomaarsen · 2026-10-26
• Tiny MoE ColBERT retrievers: 1.5M encoder to 115M model rivaling larger dense models — tomaarsen · 2026-10-26