Models Over-Attribute Being Evaluated; Private Evals Elicit Far Less Eval Awareness
dfrsrchtwts · x · 2026-10-27
Commenting on a model eval report, dfrsrchtwts highlights two observations: models over-attribute being in evaluation, and the report's private evals elicited far less eval awareness (at least 2 of 3). He also suspects the report takes Apollo's in-context scheming benchmark too literally, questioning whether GPT-6 — which reportedly beats all models — is really the least schemey model.
He adds that qualitative descriptions of model behaviour would have been more informative than marginal eval numbers.
Related event: DeepSeek Safety Eval Finds Models Over-Attribute Being Tested(2 posts)→
More from Models
- OpenAI Announces Three New Discoveries in Math, CS, and Physics — thesaraharminta · 2026-10-27
- Reddit Users Say the Ming Image Model Is Seriously Underrated — alley103 · 2026-10-27
- GPT-6.1 Terra fills form in 53.5s vs 3:21 for Sol in computer use side-by-side test — sandersted · 2026-10-27
- DeepMind staff reply mentions "Gemini 4 models" in apparent early slip — vivnat · 2026-10-27
- OpenAI math repo adds new Lean-verified papers, including an ETH refutation claim — danintheory · 2026-10-27
- OpenAI models produce Lean-verified existence proofs for QCD and Yang–Mills — danintheory · 2026-10-27