Purchasing agent in production: >95% recall, >90% accuracy, author asks how to push F1 higher
Ancient_Ad_916 · reddit · 2026-10-26
A developer shares a production agent built for their purchasing department: it automates hundreds of weekly supplier delivery emails that previously had to be manually entered into the ERP system.
Architecture:
- Ingest and clean emails into structured tables via regex parsing
- Define Pydantic classes per confirmation type
- Run pre-screened SQL against those tables for extra filtering/extraction
- Feed the email (with thread predecessors), open ERP data, regex columns, and per-supplier specifics as LLM context
- LLM either auto-commits to ERP at high confidence or flags for manual review
The system currently achieves >95% recall and >90% accuracy, but the author is stuck on pushing F1 higher and asks experienced devs for tips. A solid case study of confidence-gated human-in-the-loop design for risky enterprise automation.
More from coding & agent
- Collage Homage to Maggie Appleton's 'Planning with Agents' Talk — GrantCuster · 2026-10-26
- Feed Your Daily Tools to an Agent: A Recipe for a Workspace Built Around Your Business — EXM7777 · 2026-10-26
- Why AI filmmakers should build small tools for their slowest workflow step — lmoroney · 2026-10-26
- Kitebook: self-hosted open-source MCP server lets AI assistants hand off work to each other — ChainEnough5654 · 2026-10-26
- Watch an AI agent build a Minecraft scene block-by-block in a 6-hour timelapse — flowersslop · 2026-10-26
- Kornia author: AI agents skip library docs and read code/docstrings instead — ducha_aiki · 2026-10-26