Open DocFieldBench: Gemini 3.8 Flash scores 94.6 on 260-document extraction test
refractr · reddit · 2026-11-19
Refractr released DocFieldBench, a fully open benchmark for structured document extraction over 260 docs (invoices, bank statements, SEC filings, etc.). Scoring: +1 correct, 0 missed, -1 confidently wrong, with 1 in 7 fields deliberately absent to catch hallucinations. Results: Gemini 3.8 Flash leads at 94.6, DeepSeek v4.1 Flash 93.2 (fastest at 3.0s/doc), Mistral Large 4 91.4, Refractr 89.4 at just 0.7s/doc, GPT-5.4 mini 86.2, Llama 3.1 8B 73.9. Docs, answer key and scorer are all public.
More from Models
- Gemini Hydrogen High Reportedly Beats Claude Opus 5.5 Extra Tonight — RylanSchaeffer · 2026-11-19
- Decision model face-off: Liquid AI d1.2-ultra matches rivals at half the cost — JosephJacks_ · 2026-11-19
- Leaked Gemini 4 Neon benchmarks show sweeps across 17 evals vs Claude, GPT — nevrekaraishwa2 · 2026-11-19
- Kimi K3.1 Pro beats K3 on 14 of 17 benchmarks; pairing with Opus 5.5 halves spend — Kimi_Moonshot · 2026-11-19
- Simular box hits OSWorld v2 SOTA at 80.7% success, $1.79 per task — 8x cheaper than August — xwang_lk · 2026-11-19
- Peking University Releases Foundation Model for Measuring and Predicting Human Behaviour — PekingUniversity · 2026-11-19