Why vocab tests work for human IQ but Goodhart in model training
JoshPurtell · x · 2026-11-02
A neat analogy on eval design: human IQ batteries can include vocab tests because nobody grinds vocab specifically to game the score — but in model training such signals get Goodharted.
The key distinction: training data must remain valid under all policies that could be searched during training, whereas eval data only needs to be valid for the particular set of finished models produced by a particular procedure (ideally trained in ignorance of the evals).
More from Research
- SketchDirector: agentic layout reasoning generates complex scene sketches — GVCLab · 2026-11-02
- Thought experiment: would AI agents in a physics sim discover laws or reverse-engineer the engine? — zxpawanx · 2026-11-02
- HalachaBench sparks debate: is Talmudic Q&A testing knowledge or reasoning? — teortaxesTex · 2026-11-02
- Sebastian Raschka releases free 11-hour course on building reasoning models from scratch — AxSaucedo · 2026-11-02
- Microsoft and Berkeley open-source TokenEconomizer, cutting coding agent costs 18–31% — victor_explore · 2026-11-02
- TLA-Verus uses LLM agents to prove temporal properties of TLA+ specs — tianyin_xu · 2026-11-02