Why OCR models mix scripts: "cruel" and 残酷 have nearly identical embeddings
VikParuchuri · x · 2026-10-16
Vik Paruchuri offers the team's best explanation for script confusions: inside the model, words with the same meaning across different scripts end up with nearly identical embeddings — "cruel" sits very close to 残酷. An attached chart shows the 5 closest Chinese tokens to "cruel" in the public Chandra OCR 2 model (53,000 Chinese tokens): 残酷, 残忍, 冷酷, 粗暴, 无情的.
Related event: Datalab founder explains why VLMs confuse scripts during OCR(5 posts)→
More from Models
- NEURA launches Neuron AI, a multimodal PLA model giving robots touch, force feedback and memory — CyberRobooo · 2026-10-17
- Sentience claims per-user models: train, continually update and serve one model per person — soleio · 2026-10-17
- Four Gemini 4 Argon agents beat Minecraft in 27 hours, writing 2,783 programs — GoogleDeepMind · 2026-10-17
- Meta open-sources Muse Spark Open to mixed reception: 'trash' license, nerfed benchmark scores — gabriberton · 2026-10-17
- If new diffusion models are open-weight only, what's left for commercial use? — blackpanda8484 · 2026-10-17
- Reflection Beam OSS architecture: sliding window attention, gated attention, QK Norm — A_K_Nain · 2026-10-17