Microsoft's Taste-Bench: best frontier model scores only 59.7% on long-horizon agent decisions

microsoft · hf · 2026-09-23

Microsoft introduces "taste" — an agent's ability to make good decisions at critical forks in long-horizon tasks (which hypothesis to test, which implementation to build on) — and argues no existing benchmark measures it.

Taste-Bench

Automatically mines decision forks from agent trajectories in engineering and research tasks, using both parallel attempts at the same task and detours within a single trajectory — no human annotation needed
Each question presents a fork; the model picks a direction without seeing what follows

Findings

The best frontier model answers only 59.7% correctly
Forks whose deciding evidence appears later in the trajectory are much harder for all models
Larger reasoning budgets don't improve accuracy

Taste is trainable: distilling outcome-aware teacher judgment into a student model improves decisions on unseen tasks and lifts end-to-end success on held-out SWE-bench Pro tasks.

Original post →

Microsoft's Taste-Bench: best frontier model scores only 59.7% on long-horizon agent decisions

More from coding & agent