Tilting Pages by 2–4° Changes the Winner: BenchBench Stress-Tests Parser Benchmarks
thisguyknowsai · x · 2026-11-18
Developer raunakdoesdev set out to test his earlier claim that every document parser ranks #1 on its own benchmark.
Method: run 8 parsers across 12 benchmarks, then tilt the pages by 2–4 degrees and rerun. Six of the twelve benchmarks produced a different winner. The project is dubbed BenchBench.
Takeaway: current document-parsing benchmarks are highly sensitive to trivial image perturbations, so leaderboard rankings should not be taken at face value.
More from Research
- Author clarifies "nothing looks interesting" remark: it's about reviewing, not condescension — rishabh16_ · 2026-11-18
- Researcher laments reviewer fatigue as coding agents flood academia with incremental papers — rishabh16_ · 2026-11-18
- Stanford team predicts viral evolution and designs antibodies that neutralize future escape variants — BrianHie · 2026-11-18
- GPBam: New German Public Law Benchmark Finds Retrieval Gains Limited, Essay Quality Tracks Statutory Knowledge — aaire · 2026-11-18
- AI agents uncover first known genome-wide frameshifting in bacteria — GaryMarcus · 2026-11-18
- Google launches Antigravity Science plugin, gives 10,000 scientists free top-tier access — DynamicWebPaige · 2026-11-18