Abacus Docs vs Docling: how to eval extraction on 300 scanned contracts
Suresh935 · reddit · 2026-11-18
The author is comparing Abacus Docs and Docling for extracting renewal date, notice window, governing law, and auto-renewal clause presence from 300 scanned vendor contracts (mostly 10–40 pages).
Key discussion points:
- Without manually annotating every clause or OCR token, should the eval report per-field exact-match accuracy or reviewer minutes spent correcting each document?
- How to separate OCR misses from reasoning errors, since the two call for different fixes.
- Open call for advice on eval set design from anyone who has tested either tool on scanned contracts.
More from coding & agent
- Business owner edits a Google Doc, a daemon sends it to Claude Code to ship WordPress changes — colinmcnamara · 2026-11-18
- Production-grade AI agents are full architectures, not just prompts — goyalshaliniuk · 2026-11-18
- dusk bot launches as a free local AI agent crew that plugs into Claude, ChatGPT, Gemini or Grok — EXM7777 · 2026-11-18
- Dev builds Claude Code bot that turns X/HN AI news into 2-3 minute explainer videos — mhmazur · 2026-11-18
- Steve Yegge on AI's "vampiric effect": nonstop agent use leaves heavy users drained — bigblueboo · 2026-11-18
- Dev builds open-source Resolve-style Linux video editor Kinoklip with Claude Code — SeniorMatthew · 2026-11-18