glasshouse: a long-term memory benchmark with 2,847 questions over a 1.97M-token conversation
True_Mongoose_7073 · reddit · 2026-10-16
A memory-infrastructure company released a draft long-term memory benchmark called glasshouse: 2,847 questions over a 1.97M-token conversation in 10 languages, reported across 17 separate axes with no single headline score. Unusual axes include whether a system serves the old value after a fact changes, and how it handles contradictory stored facts.
The author admits nobody has submitted yet (including themselves), scoring costs money since it calls a judge LLM on the submitter's own key, and they're soliciting feedback from agent-memory developers on what's missing or unfair.
More from coding & agent
- Composing music with math patterns via Gemini agents in Antigravity CLI — fofrAI · 2026-10-16
- Write the Agent Loop by Hand First: What LangGraph Actually Adds — m3m3o · 2026-10-16
- $20-plan users have no idea how far AI agents have come in 10 months — realmeetjames · 2026-10-16
- Google open-sources AX, a Kubernetes-style orchestrator for billions of AI agent tasks — thisdudelikesAI · 2026-10-16
- Tencent Hunyuan open-sources Hy Memory 2.0: model, SDK and MemHub for agent memory — TencentHunyuan · 2026-10-16
- ERP•AI launches headless SaaS with $0 seats that bills machines: $1 buys 4,000 record writes — rohanpaul_ai · 2026-10-16