Disabling Web Search Unexpectedly Boosts Model Eval Scores
StefanoErmon · x · 2026-07-08
Stanford professor Stefano Ermon shared an interesting AI model failure mode: during testing, disabling the model's web search functionality actually improved its WideSearch evaluation score. This reveals potential flaws in current models regarding retrieval-augmented generation.
More from Models
- Poster says Kimi’s near-term outlook depends on a K3 base model release — _xjdr · 2026-07-27
- Kimi K3 may be strong on cyber, but token efficiency keeps it off UK AISIS — teortaxesTex · 2026-07-27
- European ChatGPT Plus users are now seeing an “Extra High” quality option — PressPlayPlease7 · 2026-07-27
- Opus 5 reportedly aces a car-racing game test on the first try — soumitrashukla9 · 2026-07-27
- Claude Opus 5 arrives at half the price and tops Frontier-Bench claims — GregCook2011 · 2026-07-27
- Open models may beat closed ones for cyber defense, researchers argue as Kimi K3 impresses — eliebakouch · 2026-07-27