As models improve, writing good evals gets harder and more important
amankhan · x · 2026-11-12
A short observation from developer amankhan: as models get better, writing good evals becomes both harder and more important — stronger models make coarse benchmarks less discriminative, turning eval engineering into the key lever for measuring real capability.
More from coding & agent
- Dell expands AI Data Platform: agents have a data-context problem, not a model problem — DavidLinthicum · 2026-11-12
- Dev's favorite AI use case: Claude monitors competitors and emails alerts — doooyle · 2026-11-12
- LLM controls vocal synth to perform Ave Maria in 4-part harmony — AAAzzam · 2026-11-12
- ArcGrav: a cross-model Agent Control Language to cut token overhead — ArchitecturalGravity · 2026-11-12
- Claude watches competitors around the clock and emails me what matters — GlenBradley · 2026-11-12
- Developers say Gemini 4 Carbon is notably better at favicon design and micro-interactions — AI_Andrew · 2026-11-12