Microsoft's Taste-Bench: frontier agents pick the better direction only ~60% of the time
omarsar0 · x · 2026-09-25
A Microsoft-led paper introduces Taste-Bench, which tests whether agents have good "taste" at decision forks in long tasks. Forks are auto-mined from parallel attempts and detours in engineering/research runs. The best model answers just 59.7% correctly; forks with late-appearing evidence are much harder, and a larger reasoning budget doesn't help. Distilling a teacher's outcome judgment into a student model improves end-to-end success on held-out tasks.
More from coding & agent
- Perplexity CEO shows agent building browser-playable 3D zombie shooter in hours with zero 3D assets — AravSrinivas · 2026-11-11
- The complete harness engineering roadmap: how OpenAI shipped ~1M lines with zero hand-written code — blaizedsouza · 2026-11-11
- Micius-Flash: 27B Multimodal Model for Hardware and Semiconductor Agents — zhiman-ai · 2026-11-11
- Viral 'the harness is the machine' thread touts unverified Claude Fable/Mythos 5.2 9-equation architecture — blaizedsouza · 2026-11-11
- Compaction creates a new failure mode for coding agents: critical bug details can vanish mid-fix — blaizedsouza · 2026-11-11
- What 40+ coding-agent harnesses do between your prompt and the model, dissected — blaizedsouza · 2026-11-11