A Simple Eval to Catch Agents That Miss Expired Reservations and Trust Cached Status

jylusdev · reddit · 2026-11-18

Reddit user jylusdev shares a compact agent eval design: a booking tool returns a reservation created at 09:00, valid until 10:30 with an exclusive end time, and the system never emits a separate event when the hold expires.

The test:

He also suggests auditing citations: they should preserve the expiry and the evaluation time. If your tool caches an interpreted status, retry without refreshing the source and check whether the cached "active" survives past the deadline — a targeted probe for agents that treat interpreted state as ground truth.

Original post →

More from coding & agent

coding & agent channel →