Agent Products Must Define "Important" First
annetgriffin · x · 2026-07-18
The author argues that knowledge extraction is inherently a part of agent product design, rather than simply throwing a vague prompt like "extract important events from call logs" at raw data.
Key takeaways include:
- AI doesn't know your definition of "important"; the objective must be productized and structured first.
- For vertical products, users expect correct results on the first try, unlike general chatbots where trial and error is acceptable.
- Examples must be used to train the system and form evaluation sets to ensure stable outputs.
- Letting an agent "figure it out on the fly" at runtime without pre-defining which steps should be automated will burn through massive amounts of tokens, ruining both cost and user experience.
The author also notes that they and their friends are tackling this exact problem across multiple products, adding that solving this at scale is extremely difficult.
More from coding & agent
- A roundup of AI agents and MCP resources, including how to evaluate agents — _jaydeepkarale · 2026-07-21
- A full course shows how to build and deploy an AI agent with OpenAI and LangChain — _jaydeepkarale · 2026-07-21
- A beginner guide to AI agents points readers to a Stanford webinar — _jaydeepkarale · 2026-07-21
- A practical guide on how to evaluate AI agents — _jaydeepkarale · 2026-07-21
- MCP is headed toward easier scale, event-driven extensions, and workable file uploads — EricBuess · 2026-07-21
- Developers debate the missing composition model for AI agents — threepointone · 2026-07-21