H Company's NeoMME: 260M encoder matches 3.75B ColQwen2.5 on ViDoRe v3 with 14x fewer params
tomaarsen · x · 2026-09-07
H Company open-sourced NeoMME, a family of 260M/800M multilingual multimodal encders built as a single bidirectional Transformer trained from scratch with a masked discrete-diffusion objective — no pretrained vision tower, no causal decoder, 16,384-token context.
Highlights
- Page-image retrieval (ColPali-style) including charts and tables, no OCR step; NeoMME-Retriever returns dense + late-interaction embeddings in one forward pass for retrieve-then-rerank
- Scores 0.523 nDCG@10 on ViDoRe v3 at 260M vs 0.524 for ColQwen2.5-v0.2 at 3.75B (14x fewer params); the 800M hits 0.556
- Encodes 51 pages/sec on an L40S at 2048x2048 input, 2x ColModernVBERT throughput
- Hierarchical token pooling + asymmetric quantization shrink late-interaction index storage from 1.5MB to 6kB per page (255x) while keeping >95% nDCG@10
- Sentence Transformers integration: MultiVectorEncoder, Matryoshka dense checkpoints (128–1792 dims, truncatedim=256), HierarchicalTokenPooling (poolfactor=2 halves page vectors)
- Same checkpoint does cross-lingual text retrieval via MeanMaxSim
All checkpoints are Apache 2.0, with a technical report and Visual RAG demo.
Related event: H Company Open-Sources NeoMME: 260M Multimodal Encoder Matches ColQwen2.5(2 posts)→
More from Models
- Gemini's agentic video understanding is off by default in the API — patloeber · 2026-09-07
- Gemini's agentic video understanding cuts tokens by 88% and costs by 66% — patloeber · 2026-09-07
- Building a neutral Codex-class harness requires running the original inner harness, dev says — joshalbrecht · 2026-09-07
- Teknium: Train on Multiple Harnesses to Keep Models Portable Across Evals — Teknium · 2026-09-07
- Reddit dev: no way to know if a script costs 20 cents or 20 dollars until it finishes — Thefounderman1 · 2026-09-07
- 41 OCR models curated into four Hugging Face collections, from manga to handwriting — vanstriendaniel · 2026-09-07