Testing Model Knowledge Cutoffs with Events
giffmana · x · 2026-07-11
This post introduces a simple **model knowledge cutoff benchmark**: the author fabricates a batch of "unexpected events" for each month and asks the models questions about them, using their awareness to reverse-engineer their knowledge update boundaries. The original post also makes an observation: models from **OpenAI and Anthropic** seem to be among the few maintaining up-to-date knowledge. Other vendors, including some Chinese AI labs, are generally at least **12 months** behind. The author notes that the highlighted blocks in the chart represent their inferred actual cutoffs.
More from Research
- Draft paper uses Markov-chain eigenfunctions to build partitions and speed up sampling — michaelchchoi · 2026-07-21
- Autoresearch proposes packaging ML runs as studies with questions, analysis, and code diffs — morgymcg · 2026-07-21
- GitHub repo adds lightweight ternary QAT for Prism-ML Bonsai models — terminoid_ · 2026-07-21
- Qdrant co-hosts a Munich meetup on search, retrieval, and agentic RAG on July 23 — qdrant_engine · 2026-07-21
- GigaChat Audio targets long-form audio grounding with timestamps across 120-minute inputs — ai-sage · 2026-07-21
- Paper models Transformer components as stochastic geometry and tests five architectures — Zhihua Liang · 2026-07-21