Tilting Pages by 2–4° Changes the Winner: BenchBench Stress-Tests Parser Benchmarks

thisguyknowsai · x · 2026-11-18

Developer raunakdoesdev set out to test his earlier claim that every document parser ranks #1 on its own benchmark.

Method: run 8 parsers across 12 benchmarks, then tilt the pages by 2–4 degrees and rerun. Six of the twelve benchmarks produced a different winner. The project is dubbed BenchBench.

Takeaway: current document-parsing benchmarks are highly sensitive to trivial image perturbations, so leaderboard rankings should not be taken at face value.

Original post →

More from Research

Research channel →