Open DocFieldBench: Gemini 3.8 Flash scores 94.6 on 260-document extraction test

refractr · reddit · 2026-11-19

Refractr released DocFieldBench, a fully open benchmark for structured document extraction over 260 docs (invoices, bank statements, SEC filings, etc.). Scoring: +1 correct, 0 missed, -1 confidently wrong, with 1 in 7 fields deliberately absent to catch hallucinations. Results: Gemini 3.8 Flash leads at 94.6, DeepSeek v4.1 Flash 93.2 (fastest at 3.0s/doc), Mistral Large 4 91.4, Refractr 89.4 at just 0.7s/doc, GPT-5.4 mini 86.2, Llama 3.1 8B 73.9. Docs, answer key and scorer are all public.

Original post →

More from Models

Models channel →