Why CER breaks down for full-page manuscript transcription: from CRNN+CTC to layout decisions
vanstriendaniel · x · 2026-09-04
vanstriendaniel explains CER made sense for line/word-level evals with CRNN+CTC models (fixed vocab, no formatting or reading order), but full-page transcription now forces dozens of stylistic decisions the ground truth fixed one particular way — all counted as reading errors. He plans to compare other eval approaches for this low-resource domain.
Related event: 0.8B Qwen Fine-Tuned on Medieval Manuscripts Sparks CER Debate(4 posts)→
More from Research
- Astra solve-rate barely improves at max compute, undercutting the 'too smart to throttle' RL theory — zainhas · 2026-09-05
- Developer runs full fruit fly connectome — all 166,700 neurons — inside Minecraft — ZeroStateReflex · 2026-09-05
- Homework for researchers: extending RoPE to tensor product representations — thomasahle · 2026-09-05
- Debate: Models Fuzzily Recall Concepts, Not Text — SAE Features vs Edit-Distance Memorization — voooooogel · 2026-09-05
- ICML Position Paper: Unlabeled Data Doesn't Mean No Human Supervision — serrjoa · 2026-09-05
- VLA-Corrector from ZJU & Alibaba DAMO lifts robot success rates while cutting policy calls — 机器之心 · 2026-09-05