New agent evals field guide draws fire: only 15 of 313 citations predate 2023
fredahshi · x · 2026-10-25
randomwalker's team released a 'field guide' paper for agent evals, arguing that as LLMs became agents, earlier best practices no longer serve and the young field needs shared guidance.
Sebgehr, while calling it cool work, pushed back: the field keeps reinventing the wheel — most 'emerging trends' were studied extensively in prior surveys (he authored one). He notes the paper's 313 citations include only 15 from before 2023, a telling sign of neglected prior literature.
More from coding & agent
- Dev launches staygreen.io, a lifetime-free uptime monitor, after balking at $330/mo white-label pricing — PawelHuryn · 2026-10-25
- Nous Research's Teknium Shows How to Plug Local vLLM Endpoints into Hermes Cloud — Teknium · 2026-10-25
- PSAISuite gives DeepSeek real PowerShell tool-calling in your terminal — dfinke · 2026-10-25
- Google AI Studio readies Managed Agents hub for inspecting autonomous agents — testingcatalog · 2026-10-25
- Codex user reports usage down 31% after reset, calls OpenAI pricing uneconomical — CtrlAltDwayne · 2026-10-25
- Non-technical builder shares 3-tool AI stack to research, design and ship full-stack apps solo — damienghader · 2026-10-25