Datalab Launches OmniParseBench and Exposes Benchmark Gaming in OCR
Datalab released OmniParseBench, a fairer OCR benchmark where it declines to rank itself first, and demonstrated that LlamaIndex ParseBench can be gamed: a 650M model reached third place after just 70 RL steps with no real OCR improvement.
2026-10-13 ~ 2026-10-13 · 3 related posts
- Episode 1: Datalab Releases OmniParseBench: 16,288 Open OCR Tests, Admits Its Own Model Isn't #1(2026-10-10, 8 posts)
- Episode 2: Datalab Launches OmniParseBench and Exposes Benchmark Gaming in OCR(2026-10-13, 3 posts)
- Datalab launches OmniParseBench, a game-resistant OCR benchmark where it doesn't rank first — VikParuchuri · 2026-10-13
- Datalab shows ParseBench is gameable: 70 RL steps lift a 650M model from 23rd to 3rd — VikParuchuri · 2026-10-13
- Surya team ships 650M OCR model after 70 RL steps, admits it's just benchmark formatting — VikParuchuri · 2026-10-13