Users who never start new chats cost 8x average — one modal prompt fixed it
Lopsided_Feedback_81 · reddit · 2026-11-02
An indie developer running a data-analyst AI assistant for small e-commerce stores (3-10 person teams) shares a real lesson: assume your users don't know how to use LLMs.
Analyzing MLflow traces, they found some users cost over 8x the average per conversation. Not because of big data pulls — these users simply never start a new conversation, piling unrelated questions (refund rates, FB campaign performance) into one monolithic, days-long thread where old context can't even hit cache.
- Context compression/truncation was tried before and clearly hurt performance; near-limit warnings alone weren't enough
- Fix: before sending each new message to the agent, a quick check decides if it's related to the conversation; if not, a slightly annoying modal asks "This seems like a new topic. Start a new chat?"
- Result: 90% "Yes" rate, and snowballing conversations dropped sharply
Takeaway: products must guard-rail non-technical users rather than expect best practices.
More from coding & agent
- Flock: A New iOS App for Running AI Coding Agents, Built on a t3code Fork — seatedro · 2026-11-03
- Vibe coding's 5-year leap: from a GPT-3 React button demo to an Opus 5.5 three.js world — bennash · 2026-11-03
- Stripe's Link Agent Wallet lands in Merge Agent Handler: agents pay without seeing card details — shensi · 2026-11-03
- Ask Your Agent to Reverse-Engineer HTTP Calls Into a CLI, Skip Slow Computer-Use — vivekhaldar · 2026-11-03
- The 'Illustrated LLM Book' repo is free: 12 chapters of runnable Colab notebooks — mdancho84 · 2026-11-03
- Google open-sources Elephant-Goldfish Model: agent skills for software design and review — adnan_hashmi · 2026-11-03