Auditing 15 document-parsing benchmarks: OmniParseBench scores 87.2, XDocParse just 5
VikParuchuri · x · 2026-11-20
Vik Paruchuri shared an audit chart scoring 15 document-parsing benchmarks on four axes (reads correctly 50, fair to every parser 25, runnable 15, separates parsers 10), out of 100. OmniParseBench (Datalab's own, bias acknowledged) leads at 87.2, followed by Real5-OmniDocBench at 79.2 and PureDocBench at 78.4; OmniDocBench v1.0–1.6 cluster at 74–77; LogicsDocBench scores just 10.6 and XDocParse 5. He considers OmniDocBench derivatives, PureDocBench and OmniParseBench the benchmarks that align with actual OCR quality.
Related event: Datalab's Vik Paruchuri Audits OCR Benchmarks and Introduces OmniParseBench(5 posts)→
More from Research
- Perception verbs alone identify fiction with 90% accuracy, decision tree shows — _akpiper · 2026-11-20
- AI2 launches Theorizer in Asta, an AI that proposes testable scientific theories from literature — allen_ai · 2026-11-20
- Cross-institutional survey asks whether mechanistic interpretability actually makes AI safer — sarahookr · 2026-11-20
- μSAM2 released: SOTA microscopy segmentation trained on 30M+ annotated cells — alxndrkalinin · 2026-11-20
- SnakeBench: frontier models still fail to plan more than 2-3 moves ahead — GregKamradt · 2026-11-20
- Most benchmarks are made by vendors chasing #1 — how to stress-test them yourself — VikParuchuri · 2026-11-20