Read-only POS agent retro: near-zero Gemini cache hits, HITL writes on hold
RedaHaloubi · reddit · 2026-09-17
A retro of a read-only POS analytics agent (route → fetch → narrate → ground, Node/TS + Vercel AI SDK): the author is holding human-in-the-loop writes until the read loop tightens, adopting pin semantics, CORE-first schemas, and evals on traces/fetch/step count/DTO coverage.
Model & cost: Gemini 2.5/3.5 Flash Lite in dev; a typical ask costs 4,840 input / 123 output tokens across 2 LLM steps in 2.3s. But Gemini implicit cache hits are just 2.2% overall — 0/96 on the current CORE-12 and 0/51 on pinned requests — so the prefix can cache but doesn't, which is why no second model call was added.
Open questions: pause with UI chips on date ambiguity vs. default window named up front, and EN+FR synonym tables on shared regexes vs. translate-for-routing for bilingual intent matching.
Related event: Read-only POS agent retrospective: cache hit rate near zero, writes on hold(2 posts)→
More from coding & agent
- Seroter's daily reads: tell agents the why, 72 trials testing if agent skills help — rseroter · 2026-09-17
- The biggest AI waste is making agents talk like humans; agent-to-agent calls could kill hold music — RichardsonDx · 2026-09-17
- Developer builds voxel Unity game in hours with GPT-6 Astra — Angaisb_ · 2026-09-17
- Stop maxing reasoning effort: iterate on low, audit on max — johnlindquist · 2026-09-17
- Dev builds fully playable battle game in six days using AI tool Astra — pvncher · 2026-09-17
- GitHub rewrote Copilot runtime in 800K lines of Rust, mostly written by AI agents — GitHub Blog AI/ML · 2026-09-17