Japanese team trains image encoder from scratch to match SigLIP2, with better CJK performance
HirokatuKataoka · x · 2026-11-04
Noetra published a tech blog detailing how it built an image encoder from scratch for its domestic multimodal LLM, aiming to remove the transparency and safety concerns of relying on closed-data models like SigLIP2.
Key points
- Training data draws on CC3M, CC12M, datacomp1b, OBELICS, MINT-1T-HTML, OmniCorpus-CC, relaion2B and the Japanese WAON dataset; interleaved documents were converted into image-text pairs by using SigLIP2 similarity to pick the sentence most relevant to each image as its caption.
- The pipeline applied missing-data removal, image size/aspect-ratio filtering, language filtering (keeping English, Japanese and Chinese—Chinese kept for kanji-driven OCR transfer, 3.4% of data, Japanese 9%), image-text similarity filtering, and dedup.
- The resulting encoder matches ViT-SO400M-14-SigLIP2 on English benchmarks and outperforms it on Japanese and Chinese ones; plugged into a multimodal LLM, it beat SigLIP2 on vision tasks.
More from Multimodal
- Can Krea2 turbo match GPT image 2.5's dreamy lens-blur look? One user's prompt saga — _gaurab · 2026-11-04
- Open-source local VFX studio runs full video pipeline under 24GB VRAM in ComfyUI — Turbulent-Barber582 · 2026-11-04
- SenseTime launches SenseNova 6.8 Flash with long-term memory and editable Blender 3D output — aftahi_ai · 2026-11-04
- Riverside Agents turns a 50-minute podcast into scored clips, hooks and a post automatically — HeyAmit_ · 2026-11-04
- Muse video model wows in real-world use, first time free at this level — op7418 · 2026-11-04
- Black hole lensing rendered with Gaussian splats: 373k Gaia stars in-browser — willeastcott · 2026-11-04