Open-source KYNEX agent loop ranks #8 on 370-doc ExtractBench, beating GPT-6 Astra with GPT-5.4
the-random-guy-2002 · reddit · 2026-11-22
KYNEX is an open-source (Apache 2.0) document-to-JSON extraction system that skips RAG entirely and runs a pure agent loop. On LlamaIndex's ExtractBench (370 enterprise docs) it ranks #8 of 53 systems, hitting 94.6 F1 with GPT-6.1 Sol and beating standalone GPT-6 Astra (91.9) using only GPT-5.4.
Architecture:
- Six tools: readpage, extracttable (with fallback chain), searchdocument (BM25 + TF-IDF), inspectpagevisually (GPT-4o vision for corrupted/image-only pages), Azure Document Intelligence OCR, and readsection for heading-based navigation
- A table of contents is injected into the system prompt so the agent knows document structure before tool calls
- A supervisor tracks filled/null schema fields and visited pages, escalating when fields stay null for 2+ rounds
- Biggest surprise: five deterministic post-processing rules (pruning non-record rows, redaction markers, table row normalization, etc.) matter enormously — without them accuracy drops hard
More from coding & agent
- Claude-of-Tanks: a tank combat sim built end-to-end by a multi-agent pipeline — Promptmethus · 2026-11-22
- Anthropic shows agents cutting tool context from 150K to 2K tokens by writing code — blaizedsouza · 2026-11-22
- AnyS1: a 4B System-1 model cuts agent workflow costs by 75–91% without losing accuracy — kalyan_kpl · 2026-11-22
- Anthropic doc: one 'you own this codebase' line lifts Claude's code consolidation from 10% to 75% — lxfater · 2026-11-22
- voxCAT shares a halftone-poster prompt template that fits two action beats of a character in one image — Promptmethus · 2026-11-22
- Late-2026 agent checklist: A2A payments, 50-branch inference, self-building RAG — blaizedsouza · 2026-11-22