7 document parsing APIs tested on 11 PDFs: no single winner
latentnoise_ · reddit · 2026-09-21
The author benchmarked 7 document parsing APIs on the same 11 real-world PDFs (35 pages, 119 calls, $9.67 total): invoices, photographed receipts, contracts, a French bank statement, noisy scans, a handwritten cheque, and a medical EOB.
Results: Claude led field accuracy at 0.991 with zero hallucinations; GPT was cheapest per correct field; Mistral OCR was fastest at 4.2s median but least accurate (0.884, 3 hallucinations); Extend hit a perfect 1.00 row F1 but tends to return null instead of guessing; Textract grabs wrong values for empty fields; GPT was fooled by handwriting on a receipt; LlamaExtract put right numbers in wrong fields.
Bonus finding: public benchmark labels in SROIE and FUNSD were themselves wrong. Takeaway: production choice should weigh errors, misses, latency, and cost together — accuracy alone doesn't decide it.
More from coding & agent
- Building an AI stock analyst with Kimi K3 + GPT-6 Astra: $450 does $60,000 of research — nevrekaraishwa2 · 2026-09-21
- LLM Judge community spent two years writing monolithic prompts until autorubric, says Delip Rao — deliprao · 2026-09-21
- Open-Source MCP Bridge Lets ChatGPT Search Local Codex History, 39/39 Sessions Verified — MeldhLLC · 2026-09-21
- V7 Labs partners with OpenAI to build AI operating system for financial work — nathanbenaich · 2026-09-21
- Dev's flight-booking MCP lets ChatGPT pick GIFs, and it chose a hilariously fitting one — Efistoffeles · 2026-09-21
- Two-pass doc pattern: 12,013 SEC pages skimmed in 32s, VLM OCR on just 2 — dl_weekly · 2026-09-21