Why self-improving agents never ship: a memory architecture with mandatory rollback

inbask · reddit · 2026-08-31

The author argues the blocker for self-improving agents isn't capability but security review: an agent rewriting its own memory unsupervised can't answer four questions — what changed, on what evidence, on whose authority, and can we take it back.

His design: the agent proposes and never applies.

Thirteen deterministic analyzers read the agent's execution history and emit typed recommendation objects, each citing evidence by content hash; no free prose allowed.
Review is a separate scope from write, self-approval is blocked against the recommending actor, and every decision carries a mandatory written reason.
Applied changes are re-measured at 1, 7 and 30 days; any regression proposes its own revert.

Three surprises:

The proposal step needs zero model calls — deterministic analyzers over typed records beat an LLM reading prose, and the real win is reproducibility (you can't A/B a stochastic proposal).
Rollback must be a precondition, not a feature: the inverse is recorded at apply time or the apply is refused, killing the whole "we'll add revert in v2" category.
Knowledge and execution history must share one store, or hash-cited evidence and the audit chain stop meaning anything. Storage is a plain SQLite file (Postgres for the server tier) with one conformance suite pinning both backends to identical semantics. Recall is 30µs p50 on an M4 Max and 361µs on a $35 Raspberry Pi 3, flat from 500 to 8,000 grains.

Limits: it improves memory, never weights; nothing applies without an explicit host grant; no daemon. Written in Rust, dual MIT/Apache, repo linked in comments.

Original post →

Why self-improving agents never ship: a memory architecture with mandatory rollback

More from coding & agent