Why CER breaks down for full-page manuscript transcription: from CRNN+CTC to layout decisions
vanstriendaniel · x · 2026-09-04
vanstriendaniel explains CER made sense for line/word-level evals with CRNN+CTC models (fixed vocab, no formatting or reading order), but full-page transcription now forces dozens of stylistic decisions the ground truth fixed one particular way — all counted as reading errors. He plans to compare other eval approaches for this low-resource domain.
Related event: 0.8B Qwen Fine-Tuned on Medieval Manuscripts Sparks CER Debate(4 posts)→
More from Research
- Aviation tracks every part's provenance — agents can reason over that graph — colinmcnamara · 2026-10-21
- SFT details: instruction/reasoning/code/math mix, synthetic + human traces, no epoching — TimDarcet · 2026-10-21
- Mid-training recipe disclosed: 32k→131k context extension, dropping logit distillation for tokens — TimDarcet · 2026-10-21
- Mathematician Daniel Litt concedes his AI bet, but says AI won't solve math — ziv_ravid · 2026-10-21
- AI drug discovery reality check: 'give me everything that binds,' then filter for toxicity — burny_tech · 2026-10-21
- PyTorch details KTIR and Dataflow Scheduler behind IBM's Spyre accelerator backend — PyTorch · 2026-10-21