Post-mortem: why in-memory agent state graphs failed in 24/7 production, and the disk-backed fix
Remarkable_Plant7820 · reddit · 2026-09-04
After running autonomous LLM agents around the clock, the author details three failure modes of in-memory state graphs (LangGraph/CrewAI style): context-window degradation driving cost and prompt drift, async exceptions corrupting the whole runtime graph, and total state loss on crashes.
They open-sourced Samantha with the fixes:
- Disk-backed memory: knowledge lives in structured Markdown files; agents flush state after deliverables and reload clean context
- Atomic file-based inter-agent IPC with inbox directories, decoupled from the memory loop
- Ephemeral, scoped worker lifecycles tracked via file plans and cleanly reaped
- Engine cascading fallback (Opus→Sonnet→Haiku, Pro→Flash) for rate limits and downtime
Code and architecture docs on GitHub.
More from coding & agent
- Your coding agent forgets its rules: 7 paper-backed fixes for compaction loss — victor_explore · 2026-09-04
- Every's consultants answer 33 AI adoption questions from 400 executives — every · 2026-09-04
- Meeting Transcript Extraction Has Plateaued: Where to Improve Quality Beyond Prompt Tweaks — grantnlee · 2026-09-04
- 2,200 Calls, $200 a Day, Zero Cost Attribution: Multi-Agent Run Accounting Has No Standard Answer — mrtrly · 2026-09-04
- Judge at OpenAI's Singapore Sites Hackathon shares how to stand out when everyone builds with agents — gabrielchua · 2026-09-04
- Dev builds local markdown-based long-term memory for AI agents, used daily for 4 months — Rudy_PH · 2026-09-04