Explosion's Vibe NLP workflow beats GPT 5.6 Sol on NER data at 1/25th the cost with fewer errors
honnibal · x · 2026-11-23
Explosion founder Matthew Honnibal published a "Vibe NLP" workflow where a coding agent like Claude Code drives a multi-step pipeline to cheaply produce high-quality NER training/eval data using open-weight models.
How it works:
- Multiple small open-weight LLMs/NER models annotate in parallel on a Kubernetes-based execution environment, with the agent following a template;
- The agent develops correction rules and gazetteers, and merges annotations via a consensus algorithm;
- Humans only answer binary-decision questions on examples where consensus is uncertain — 10–20 minutes of annotation on the most uncertain cases.
Results: on two small corpora (tools/tech mentions in HN posts; news text with PERSON/ORG/PRODUCT/GPE), the consensus of 3–5 open-weight models tied GPT 5.6 Sol on one task and beat it on the other at 1/25th the runtime cost; after light human review, the data had 50–60% fewer errors than GPT 5.6 Sol's output on both tasks.
Aim at teams needing domain NER data who find frontier models "almost good enough but too expensive."
Related event: spaCy Creator Open-Sources 'Vibe NLP' Pipeline for Low-Cost NER Data(2 posts)→
More from coding & agent
- Only-Knowing-Code Developers at Risk as the Agentic Software Factory Reshapes Engineering — Pavan_Belagatti · 2026-11-23
- MCP Tool Gateways Should Fail Loudly: A Plausible Answer From a Missing Dependency Is the Real Bug — Emergency-Tutor-7 · 2026-11-23
- Codex Can't Route PCB Traces: Devs Still Lack a Workable AI Tool for Hardware's Last Mile — goldvenetianmask · 2026-11-23
- Uncensored 26.4B model runs on a 12GB Mac at 85 tok/s with 8 parallel agents via Locals — SumitGup · 2026-11-23
- spaCy creator Honnibal shares a cheap-LLM ensemble recipe for building NER data with coding agents — honnibal · 2026-11-23
- 398M tokens, 98% cached: how one dev ran 12.5h of agent game dev on 3% of a $200 Claude plan — Yamapama · 2026-11-23