OpenAI's Kepler Agent Processes 580PB Daily, Cuts Query Time to 90s
Al_Grigor · x · 2026-08-26
OpenAI's Data Productivity team published a case study on Kepler, an internal AI data agent built on OpenMetadata serving 3,500+ employees. It enables self-service analytics across 70,000 datasets.
Key Results:
- Reduces data retrieval time from days to under 90 seconds.
- Processes 580+ PB daily with 15x faster queries.
- Every answer shows reasoning and enforces strict permissions.
Technical Implementation:
- Grounds answers in governed metadata (schemas, lineage, query history) for accuracy.
- Uses a six-layer context model and compounding memory.
- Enforces permissions at every layer, from retrieval to chain-of-thought.
More from coding & agent
- Dev bills $20,000 for an app built with $200 of Claude Code — IgorCarron · 2026-10-15
- Dan Shipper: run product and labs in parallel, even with a labs team of one — danshipper · 2026-10-15
- Gemini 4 Argon wrote 5,000+ lines of pure Python to build its own ray-tracing 3D engine — aitrendz_xyz · 2026-10-15
- Minecraft fully ported from Java to Rust, file by file, server and client included — Vjeux · 2026-10-15
- mitsuhiko builds a full Python IDE for iOS in an afternoon with new AI models — mitsuhiko · 2026-10-15
- How to get Gemini 4 Argon to build games that actually look decent — chongdashu · 2026-10-15