Why CER still matters for manuscript transcription models, despite its limits
wjb_mattingly · x · 2026-09-04
In a follow-up on his 0.8B Qwen 3.5 medieval-manuscript finetune, the author explains CER errors on the model are mostly understandable stylistic/orthographic issues, but CER was instrumental for tracking performance and conforming to CATMuS guidelines — while agreeing evaluation has gotten more complex.
Related event: 0.8B Qwen Fine-Tuned on Medieval Manuscripts Sparks CER Debate(4 posts)→
More from Research
- Astra solve-rate barely improves at max compute, undercutting the 'too smart to throttle' RL theory — zainhas · 2026-09-05
- Developer runs full fruit fly connectome — all 166,700 neurons — inside Minecraft — ZeroStateReflex · 2026-09-05
- Homework for researchers: extending RoPE to tensor product representations — thomasahle · 2026-09-05
- Debate: Models Fuzzily Recall Concepts, Not Text — SAE Features vs Edit-Distance Memorization — voooooogel · 2026-09-05
- ICML Position Paper: Unlabeled Data Doesn't Mean No Human Supervision — serrjoa · 2026-09-05
- VLA-Corrector from ZJU & Alibaba DAMO lifts robot success rates while cutting policy calls — 机器之心 · 2026-09-05