Datalab founder explains why VLMs confuse scripts during OCR

On October 16, Datalab founder Vik Paruchuri published an investigative thread examining a rare failure mode in VLM-based OCR: 'script confusion', where the model outputs a word with the correct meaning or pronunciation but in the wrong writing system. A typical example: an illegible English word 'vain' on a hymn page was output as the Chinese character '罪', which happens to be the Chinese equivalent of 'vain'.

Confirmed

• Paruchuri's team's best explanation: within the model, embeddings of synonymous words across different writing systems are nearly identical—for instance, 'cruel' and '残酷' sit very close in vector space, and this overlap is visible in the Chandra OCR 2 model at around 5.3 (presumably a layer/dimension position; the source only gives the number).
• When page text is illegible and evidence is scarce, the model's language priors override the page evidence, so it outputs a synonym in another writing system at the wrong position.
• After testing multiple models one by one, Paruchuri confirmed that this 'right word, wrong alphabet' confusion occurred in every tested model, including Gemini Flash 3.8 and Claude Opus 5.5.

Why it matters

• The phenomenon is not a defect of a single model but is prevalent across models, pointing to a shared mechanism in VLM internal representations: semantic embeddings are not sufficiently sensitive to writing systems.
• For high-fidelity OCR scenarios (historical texts, handwriting, low-quality scans), understanding the cause of such errors is a key clue to improving models.

2026-10-16 ~ 2026-10-16 · 5 related posts

Full story(2 episodes)→

• Episode 1: Datalab founder explains why VLMs confuse scripts during OCR(2026-10-16, 5 posts)
• Episode 2: Datalab's Chandra OCR 2.5 Cuts Script Confusions 7x with New Training Method(2026-10-16, 2 posts)

• [source] Why VLMs sometimes output 罪 instead of 'vain': an OCR investigation — VikParuchuri · 2026-10-16
• OCR models keep writing the right word in the wrong alphabet — and frontier models do it too — VikParuchuri · 2026-10-16
• [source] Script confusion bug confirmed in Gemini Flash 3.8 and Claude Opus 5.5 — VikParuchuri · 2026-10-16
• Why OCR models mix scripts: "cruel" and 残酷 have nearly identical embeddings — VikParuchuri · 2026-10-16
• [source] 'cruel' sits next to 残酷: embeddings explain VLM script confusions — VikParuchuri · 2026-10-16