Hindsight ships 0.8B model for agent memory decisions: 2.5x cheaper, 1.9x faster
victor_explore · x · 2026-11-04
Hindsight (vectorize-io), an open-source agent memory project with 58.6k GitHub stars, shipped a 0.8B model that takes over the high-frequency "is this new? does it matter?" judgment calls in memory pipelines.
- Core idea: every dedup/relevance check in agent memory is a full LLM call — a hidden bill. Replacing those with the small model on top of gemini-3.8-flash cuts cost 2.5x and model time 1.9x while scoring better
- Benchmarks: across 12 models (gemini, claude-haiku, gpt, deepseek, glm, qwen, etc.), adding hs-decide-1-flash moves most models into the "best value" zone (cheaper and better)
- MIT licensed, with Docker/Helm deployment, CLI, client integrations, and system eval suites
More from coding & agent
- Quota reset night: Codex back to 100% while Claude shows just 10% left — rudrank · 2026-11-04
- Self-improving software is inevitable: closing the agent-observability loop — bibryam · 2026-11-04
- Building a 7-GPU 132GB VRAM local inference rig: consolidate on EPYC or split across two systems? — infectiousstupidity · 2026-11-04
- Founder of 50-person, 12-country team says he talks to AI agents more than humans — Scobleizer · 2026-11-04
- Pi launches a minimal agent harness; Zammoth adds AI engineer to Mac music app — solyarisoftware · 2026-11-04
- Reply to Karpathy goes viral again: devs who refuse AI coding will be replaced first — CtrlAltDwayne · 2026-11-04