Builder shares a 17k-memory Claude harness with Postgres retrieval and nightly judging

supermegasaurusrex · reddit · 2026-07-27

A builder shared a detailed write-up of a custom Claude harness that uses the Claude Agent SDK plus a Postgres-backed memory layer.

Scale and setup

How the memory system works

Nothing is preloaded: memories stay in the database until relevant.
Three recall points: at request start, before risky actions, and when the topic changes.
Three retrieval lanes: vector similarity, PostgreSQL full-text search, and entity-based retrieval, fused with reciprocal rank fusion.
Importance ranking: corrections, preferences, and key decisions outrank casual remarks.
Validity windows and supersession: old facts can expire and be replaced by newer ones.
Write-time deduplication: highly similar updates modify an existing memory instead of creating duplicates.
Entity extraction on every write creates an explicit relationship layer.
Project isolation keeps memories from leaking across projects.
Automatic decay and pruning removes stale or low-value material weekly.
Nightly LLM judge scores retrieval quality and a regression gate catches quality drops.
Manual feedback lets the user mark memories useful or not useful.

Limitations

Every write needs both an embedding call and an entity-extraction call.
Every recall adds a database round trip.
Extraction is noisy and still needs pruning.
The LLM judge helps with regressions but is not a substitute for a real external benchmark.

The author says the system works well for them and a client, but they want outside feedback before adding more complexity.

More from coding & agent