Most models mention hidden tests on MaxxBench, hinting at possible training contamination
dejavucoder · x · 2026-11-11
@loresjpg observed that most models mention hidden tests on MaxxBench, possibly a recognition behavior or training artifact (e.g., whether models actually use mini-swe-agent outside benchmarks); frequency varies across models. dejavucoder is investigating this "benchmaxxing" behavior, with the latest DeepSeek 4.1 checkpoint pending.
More from Models
- Google open-sources Harness², an agent that rewrites its own setup per task — Arindam_1729 · 2026-11-11
- Dev dumps Codex desktop app over quality issues, falls back to codex-cli — tekbog · 2026-11-11
- Why do old models get dumber before launches? Lab party, says joking X post — glcst · 2026-11-11
- Grok Bot now detects hidden prompt injections, warns users and refuses to follow them — XFreeze · 2026-11-11
- Shanghai Institute Open-Sources Nex-N2.6, a 321B Multimodal Agentic Model — victormustar · 2026-11-11
- Nex-N2.6 ships open FP8 weights (321B, GLM-5.3-Flash based), cuts reasoning loops 16%→1.6% — Physical_Hat4022 · 2026-11-11