Document extraction bake-off: all methods rerun same window with unified scoring, code open-sourced
VikParuchuri · x · 2026-09-17
Vik Paruchuri details the methodology behind his document extraction evaluation:
- Sources: documents drawn from ExtractBench (LlamaIndex), LongExtractionBench (Reducto), LongArray (Extend), and their own benchmark
- Fairness: every method was run simultaneously within the last 3 weeks using the best available settings at the time; the delay was to tighten scoring rules
- Consistent rules: unified criteria for null fields and row matching
- Coming soon: a way to visualize predictions plus a leaderboard
Code and data are publicly released.
Related event: Datalab Releases OmniExtractBench to Fix Document Extraction Benchmarks(4 posts)→
More from Research
- ECCV seethes as GenCeption shows video models can swallow traditional CV research — CSProfKGD · 2026-09-17
- Researcher tells Bram Cohen: fluid blow-ups are inherently unstable and uncontrollable in practice — snikolov · 2026-09-17
- Researchers salute Elad Hazan as his optimization work shapes the field — HazanPrinceton · 2026-09-17
- Elad Hazan writes personal history of online convex optimization and its people — HazanPrinceton · 2026-09-17
- TMLR Quizzed 10 Desk-Rejection Authors; Most Couldn't Explain Their Own Papers — hihey54 · 2026-09-17
- New DiD sensitivity framework explains how pre-trend violations arise from confounders — analisereal · 2026-09-17