NeoMME: single-tower multimodal multilingual encoders launched, 260M tops ViDoRe v3 under 800M

NeoMME is a family of 260M and 800M multimodal native multilingual encoders released by Hcompany on Hugging Face. Rather than adapting a large generative VLM for representations, the team trained a single bidirectional Transformer from scratch that natively handles both text and images, is multilingual, and is designed for speed. The 260M model leads the sub-800M category on the ViDoRe v3 retrieval benchmark, and the team also fine-tuned downstream models on top of it.

Confirmed

• Single-tower architecture: one bidirectional Transformer processes multilingual text tokens and raw image patches, requiring no pretrained vision tower, text encoder, or decoder, with 16384-token context.
• Pretrained from scratch with a masked discrete diffusion text objective.
• Two sizes, 260M and 800M, released on Hugging Face (official Hcompany post m6).
• The 260M model tops the sub-800M tier on ViDoRe v3.
• Downstream fine-tuning was performed on the encoder.

Unconfirmed

• Attribution varies across posts: antoinechaffin relayed it as a LightOn team release and later as a release by Liquid AI newcomer Aurélien Luciani; another post credits Tony Wu and Aurélien Lucet; the original release post (m5) cites researcher tonywu71 with the Hcompany account. Official channels should be treated as authoritative.

Why it matters

• NeoMME shows an alternative to extracting representations from generative VLMs: small, fast encoders trained natively multimodal, well suited to latency-sensitive multilingual multimodal retrieval.
• A 260M model leading larger peers on ViDoRe v3 suggests native multimodal pretraining with a diffusion objective is competitive at small scale, offering a new lightweight option for multimodal retrieval.

2026-09-03 ~ 2026-09-04 · 7 related posts

• NeoMME: single-tower multimodal encoder leads sub-800M models on ViDoRe v3 visual retrieval — _reachsumit · 2026-09-03
• [source] Hcompany Releases NeoMME: Multimodal-Native Multilingual Encoder with Masked Discrete Diffusion — Hcompany · 2026-09-03
• NeoMME: single-tower multimodal multilingual encoders beat sub-800M rivals on ViDoRe v3 — CShorten30 · 2026-09-03
• LightOn ships NeoMME: natively multimodal encoders at 260M/800M with no vision tower — antoine_chaffin · 2026-09-03
• NeoMME: From-Scratch Multimodal Encoders at 260M/800M With No Vision Tower — CShorten30 · 2026-09-03
• [source] NeoMME: 260M/800M multimodal-native multilingual encoders drop the vision tower — antoine_chaffin · 2026-09-04
• Liquid AI releases NeoMME: from-scratch native text-image encoders at 260M and 800M — antoine_chaffin · 2026-09-04