GPT-5.4 with open-source KYNEX harness outscored standalone GPT-6 Astra on ExtractBench: 92.97 vs 91.9 F1
the-random-guy-2002 · reddit · 2026-11-22
KYNEX, an open-source (Apache 2.0) document-to-JSON extraction harness, posted verified ExtractBench results (LlamaIndex's benchmark: 370 enterprise docs, 4,869 pages, 67 doc types):
- GPT-5.4 through KYNEX: 92.97 F1, beating standalone GPT-6 Astra (91.9) — last-gen model with the right tooling beat next-gen without it
- GPT-6.1 Sol through KYNEX: 94.6 F1, rank 8 of 53 systems, only 0.7 F1 from Codex on the same model
- The harness gives the LLM six tools (page reader, table extractor, OCR, vision, full-text search, Azure Doc Intelligence) plus a supervisor tracking extracted vs missing fields
- Five post-processing rules, each with ablation data: page-wise enumeration, non-record pruning, redaction markers, structured table rows, OCR fallback for image-only pages; one rule alone recovered 0→0.97 F1 on the worst-case document
- All three entries verified by LlamaIndex maintainers
More from coding & agent
- Claude-of-Tanks: a tank combat sim built end-to-end by a multi-agent pipeline — Promptmethus · 2026-11-22
- Anthropic shows agents cutting tool context from 150K to 2K tokens by writing code — blaizedsouza · 2026-11-22
- AnyS1: a 4B System-1 model cuts agent workflow costs by 75–91% without losing accuracy — kalyan_kpl · 2026-11-22
- Open-source KYNEX agent loop ranks #8 on 370-doc ExtractBench, beating GPT-6 Astra with GPT-5.4 — the-random-guy-2002 · 2026-11-22
- Anthropic doc: one 'you own this codebase' line lifts Claude's code consolidation from 10% to 75% — lxfater · 2026-11-22
- voxCAT shares a halftone-poster prompt template that fits two action beats of a character in one image — Promptmethus · 2026-11-22