Read-only analytics agent on Flash Lite: 2.2% cache hit rate, full cost data
RedaHaloubi · reddit · 2026-09-17
Developer RedaHaloubi updates his read-only POS analytics agent (route→fetch→narrate→ground, Node/TS + Vercel AI SDK): after community feedback he parked HITL writes — preview ≠ confirm, idempotency + optimistic locks, fail closed near mutations — and won't add mutate tools until the read loop tightens.
Model & cost: still Gemini Flash Lite (cheap, only needs tool selection + small JSON reads + bilingual narration, not a reasoner). Typical ask (50 intent turns, one store, Redis miss): 4,840 input tokens, 0 Gemini cached input, 123 output tokens, 2 LLM round trips, 1 tool call, 2.3s wall time. Across n=229 runs, implicit cache hit only 2.2% overall, 0/96 on CORE-12, 0/51 on pinned — the prefix can cache but isn't, so he skipped a second model call.
Two open questions: ① dates — when the phrase table misses ("hier", "this month vs last"), pause with UI chips or keep defaults and name the window in the first sentence? ② bilingual routing — English keyword pins miss French queries; he's leaning toward EN+FR synonym tables on the same regexes, not translate-for-routing (mangles $/%). Full cost table included.
Related event: Read-only POS agent retrospective: cache hit rate near zero, writes on hold(2 posts)→
More from coding & agent
- Typesafe's jev ships a live, playable demo for developers — holdenmatt · 2026-09-17
- TypeSafe AI cookbook: line-by-line semantic search scoring 218 lines in one request — hackgoofer · 2026-09-17
- 'Vibe automating': offloading admin work — lead scoring, reporting, email workflows — to AI — aronkor · 2026-09-17
- Why AI coding assistants keep generating deprecated functions, and how devs cope — sharificles · 2026-09-17
- Bend2 launches tomorrow: Victor Taelin's 'post-AGI' language ditches interaction nets for speed — Birchlabs · 2026-09-17
- Devin's Fable 5.1 + SWE-2 Fusion Mode Runs 3+ Hours on Just 1% of Usage, Early User Reports — CtrlAltDwayne · 2026-09-17