State Storage Solutions for Databricks Managed Agents
CommitteeImmediate66 · reddit · 2026-07-13
The author reflects on their approach to storing state in Databricks managed agents. The most valuable advice was categorizing agent state into three types: runtime temporary scratchpads, persistent user/workspace state, and append-only audit history; mapped respectively to Redis, Postgres, and permission snapshots/audit logs. This layered approach fundamentally shifted their understanding of the problem. Ultimately, they only implemented the second category: storing conversation history and user preferences in **Lakebase** (managed Postgres). This is because the current use case doesn't require sub-millisecond hot caching, nor are there compliance mandates for immediate immutable audit logs. For requirements like "fetch a user's latest 20 messages and preferences," a relational database is better suited than scanning Delta tables, and keeping it in Lakebase means staying within the Databricks ecosystem without deploying a separate cloud Postgres. Technically, the agent runs as a Databricks App connecting directly to Lakebase Postgres; authentication uses OAuth rather than static connection strings or passwords, with the app fetching tokens to connect, avoiding hardcoded database credentials in configurations.
More from coding & agent
- Codex turns out 123 screensavers in one playful batch — intellectronica · 2026-07-21
- Grok Build adds `grok doctor`, resumable sessions and remote image paste — mark_k · 2026-07-21
- Autoresearch proposes packaging ML runs as studies with questions, analysis, and code diffs — morgymcg · 2026-07-21
- CHAP defines approvals, handoffs, and audit logs for human-agent workflows — DeliveryTechnical199 · 2026-07-21
- The author says Codex reached 20x and is now debugging spec decoding on a hybrid parallel setup — TheZachMueller · 2026-07-21
- Axcess adds an MCP connector for WCAG accessibility checks that scanners miss — modelcontextprotocol · 2026-07-21