Marker author builds new document extraction benchmark, fixing major scoring flaws in existing ones
VikParuchuri · x · 2026-09-17
VikParuchuri (author of the Marker document parsing tool) released a new evaluation method for document extraction, pooling documents from ExtractBench (LlamaIndex), LongExtractionBench (Reducto), LongArray (Extend), and their own benchmark, with consistent rules for null fields and row matching.\n\nThey found dozens of significant errors in existing benchmarks — e.g., the same number of table mistakes can score either 100% or 0% on LongExtractionBench. Code and data are open-sourced.
Related event: Datalab Releases OmniExtractBench to Fix Document Extraction Benchmarks(4 posts)→
More from Research
- Researcher tells Bram Cohen: fluid blow-ups are inherently unstable and uncontrollable in practice — snikolov · 2026-09-17
- Researchers salute Elad Hazan as his optimization work shapes the field — HazanPrinceton · 2026-09-17
- Elad Hazan writes personal history of online convex optimization and its people — HazanPrinceton · 2026-09-17
- TMLR Quizzed 10 Desk-Rejection Authors; Most Couldn't Explain Their Own Papers — hihey54 · 2026-09-17
- New DiD sensitivity framework explains how pre-trend violations arise from confounders — analisereal · 2026-09-17
- dml.sensemakr R package released: nonparametric DiD sensitivity analysis with DML — analisereal · 2026-09-17