Datalab Finds Half of Document Parsing Benchmarks Unreliable

Datalab (Marker/olmOCR) tested 15 document parsing benchmarks with 7 open models and found roughly half of rankings diverge from real error rates, including 68 mis-scored pages out of 200 in olmOCR-bench.

2026-11-05 ~ 2026-11-05 · 2 related posts

• Datalab tested 15 document-parsing benchmarks: about half misrank parsers vs real errors — VikParuchuri · 2026-11-05
• Datalab finds judging errors on 68 of 200 pages in olmOCR-bench, shares 5 checks — VikParuchuri · 2026-11-05