The Tasteful Agent: Measuring and Improving Taste in Long-Horizon Tasks
Wenbo Pan, Zhichao Liu, Shujie Liu, Jingying Zeng, Chin-Yew Lin, Xianfeng Tang, Yan Lu, Qi He, Xiaohua Jia
cs.AI
2026-09-22
Taste-Bench mines 502 decision forks from agent traces. Best model: 59.7% both-orders correct. Distilled Qwen3.6-27B lifts SWE-bench Pro success from 14.6% to 33.7%.
On a long software or research run, the choice of which implementation to keep or which hypothesis to test spends the rest of the budget. A bad choice often looks reasonable at the time. The cost shows up much later. Existing agent benchmarks report whether the task finished. They do not report whether those forks were good. Human labels are expensive and do not travel across domains.
Researchers at City University of Hong Kong, an independent collaborator, and Microsoft call this ability taste: picking, before the outcome is visible, the direction that later evidence supports. Taste-Bench turns trajectories that agents already ran into labeled questions, with no expert annotation at mining time.
A decision fork is the point where two attempts share a prefix and then split. The evaluated model sees the task, the prefix, and two neutrally worded candidates. Everything after the fork is hidden. The label is the later test result or experiment score. The better side is the supported candidate.
Questions are mined from two kinds of traces. Parallel forks pair a passing attempt with a failing attempt on the same task and cut them at the divergence. Detour forks sit inside one run: the agent walks into a wall, abandons that direction, and recovers. The generator is GPT-5.6 Sol. Two filters then drop items that judges can solve from the candidate wording alone, and items whose label the judges reject after seeing the full record. Of 4,657 proposed forks, 10.8% survive, leaving 502 questions: 390 engineering, from 2,677 GPT-5.4/5.5 rollouts on SWE-bench Pro, and 112 research, from 1,132 MALT runs on RE-Bench and HCAST.
Scoring fights position bias. A question counts as correct only when both candidate orders are answered correctly, so chance is 25%. A human review of 100 sampled items finds 170 of 172 explicit A/B judgments matching the mined label (98.8%).
Training uses Qwen3.6-27B as the student. The teacher is the same frozen base with a short demonstration of the supported candidate in context, and it writes reasoning plus a choice. The student sees only the question and matches the teacher with a token-level KL. The 390 engineering items are split into two task-disjoint folds. After reasoning distillation, a calibration pass trains the student's own traces on the final choice.
Fourteen frontier models. The headline Average weights research and engineering 1:1. GPT-5.6 Sol scores 59.7%, GPT-5.5 59.5%, Claude Opus 5 55.5%, Grok 4.5 54.6%. Chance is 25%. Detour forks are harder than parallel forks in both domains, and that gap is larger than the domain gap.
Accuracy falls as the time horizon grows. The 14-model mean drops from 62.3% on in-prefix forks to 21.0% on more-work forks, near chance. Raising reasoning effort from lowest to highest changes GPT-5.6 Sol by −0.2 points and Luna by +2.2. Both models spend the most reasoning tokens on more-work items, where accuracy is worst.
Taste-Bench Average correlates with SWE-bench Verified at Pearson r = +0.63 (R² = 0.39). On the engineering subset the correlation is only r = +0.37. The four best Verified models sit within 4.0 points of each other there and 10.7 points apart on Taste-Bench Average.
After distillation the student scores 47.9% both-orders-correct on held-out tasks, against 30.0% for the base, a 17.9-point gain. On 41 held-out SWE-bench Pro tasks a fixed Qwen3.6-27B executor goes from 14.6% success with no advice to 33.7% with student advice, against a 39.0% ceiling with always-correct advice. Those forks were mined from earlier runs on the same tasks, so the advice is in the prompt before the new run starts. The executor then finishes the task on its own.
Models that look tied on SWE-bench can be more than ten points apart on fork judgment. Extra reasoning budget barely moves the number. What is missing is evidence that appears later, not more tokens now. For people who train agents, the traces are already supervision: the teacher sees the ending, the student sees the fork, LoRA on one A100 for about two hours.
The practical shape is an advisor. Keep the executor fixed and write the fork call as "avoid this, take that." 19.1 points on 41 tasks is real. The protocol still injects forks mined from old runs. It does not detect a live fork in a new trajectory.
Labels bundle judgment with execution quality and environment noise. Filters only drop the obvious mismatches. The research cell is 112 questions from 47 tasks, and parallel research covers 8 tasks, so variance will be high. The item generator is GPT-5.6 Sol, which also tops the leaderboard. Other judges run the filters, but generator bias remains. End-to-end advice comes from forks mined on prior attempts at the same tasks. A new run may never hit those forks, and there is no live fork detector. The teacher sees the answer during distillation. Leakage control is structural (closed two-way choice, no copied demonstration span), with no external probe for zero leakage. The paper has no Limitations section; these caveats sit in the experimental setup.