VAGOsolutions releases MoE retrievers as small as 1.5M parameters, Apache 2.0
tomaarsen · x · 2026-10-26
tomaarsen highlights VAGOsolutions' SauerkrautLM-MoEdernColBERT collection: three Apache 2.0 multilingual multi-vector retrievers at just 1.5M, 27M and 115M parameters, covering German, English, Spanish, Dutch, French, Italian and Portuguese. The 1.5M model has only 0.88M active parameters per token — a remarkably small MoE embedding model. All are on Hugging Face with GGUF and CoreML builds for on-device use.
Related event: VAGOsolutions releases tiny 1.5M-115M MoE ColBERT retrieval models(10 posts)→
More from Models
- China Telecom's XingChen lab open-sources Xing4.0-29B-Intent, a 29B MoE with 4B activated params — FellMentKE · 2026-10-26
- Deleted vLLM PR Hints Qwen4-27B Uses 4-Way Parallel Residual Streams (Unconfirmed) — Ok-Shower7286 · 2026-10-26
- a16z: OpenRouter token demand up ~27x in a year, open-weight models now serve 72% — TheZachMueller · 2026-10-26
- Running a 125B model on 8GB VRAM with Strata: 900 t/s prompt processing, 30 t/s generation — BleynSpecnaz · 2026-10-26
- More OG Hugging Face profiles to follow: local AI, diffusion, kernels and transformer.js — ariG23498 · 2026-10-26
- 9 Hugging Face profiles worth following, from TRL agents to vLLM kernels — ariG23498 · 2026-10-26