LlamaIndex launches ParseBench-2: document parsing benchmark scoring 330+ error types by severity
llama_index · x · 2026-11-13
Jerry Liu of LlamaIndex announced ParseBench-2, their most ambitious document parsing benchmark yet.
Key points:
- 330+ production parsing error types with painstakingly assigned severity scores, each metric tuned to align closely with human judgment
- Criticizes existing benchmarks: brittle deterministic metrics (OmniDocBench) or LLM-as-a-judge (ParseArena), which are coarse, stochastic, and biased by prompt+model
- 4.3k+ document pages, double ParseBench-1
- New forms dimension plus much better ground-truth across all dimensions
More from Research
- Designing IL-36 miniprotein binders in a day with Codex and a few dollars of GPU — CatAstro_Piyush · 2026-11-13
- Dual-agent AI harness optimizes chemistry codes, cutting TDDFT cost by up to 94% — CatAstro_Piyush · 2026-11-13
- TLAPS-proof LLM serving engine: 516,973 obligations at just 0.18% latency overhead — tianyin_xu · 2026-11-13
- Single-constraint KL minimization reduces to 1-D bisection on the dual — caglar_ee · 2026-11-13
- AI-Simulated Survey Matches Real Human Rankings: Spearman r=0.886 in Head-to-Head Match-ups — soumitrashukla9 · 2026-11-13
- Professor Uses AI Simulation to Pick His Executive Headshot from Six Looks, Biker Jacket Included — soumitrashukla9 · 2026-11-13