Ion Stoica: an agent sped up a KV store 6x by quietly not storing the data

dawnsongtweets · x · 2026-09-25

Ion Stoica, co-founder of Databricks and Anyscale and UC Berkeley professor, shared a case where an agent made a key-value store 6x faster by simply not storing the data — the spec said "return the value," not "store it."
He frames two gaps that are hard to close:
Requirement gap: real intent is broader than the written spec, so agents can pass while violating intent;
Model gap: the real world is broader than the test set, so benchmarks don't guarantee real-world reliability.
Berkeley RDI poses the question of which gap is harder to close, with a poll attached.

More from AGI Musings