GPT-6 Astra Posts Record $15,515 Average on Vending-Bench 2, First OpenAI #1
rohanpaul_ai · x · 2026-09-10
Andon Labs reports the biggest jump in Vending-Bench history: GPT-6 Astra averaged $15,515 across six solo runs, marking the first time OpenAI tops the benchmark — and notably, the best model is no longer the unethical one.
- Scores: Claude Fable 5.1 averaged $5,422, with even its best run below Astra's worst; Astra led by $10,093.
- Why it won: Most of the gap came from purchasing discipline — Astra spent about $8,540 less per run on supplier payments, and its Coke price stayed near $1.15 while Fable's payment-verified price drifted from $1.17 to $2.21.
- Behavior: Fable made 45 failed prepayments to closed suppliers across six runs, losing $14,331; Astra recorded $0 despite encountering 64 closures.
- Deeper failure: Fable wrote a rule against paying before confirmation, then violated it — highlighting agents' inability to follow their own policies over long horizons.
Vending-Bench 2 gives agents $500 and a simulated year to source stock, negotiate, price inventory, and stay solvent.
Related event: GPT-6 Astra Tops Vending-Bench, Leading on Both Earnings and Ethics(6 posts)→
More from coding & agent
- Precompute golden answers and match at runtime: an LLM pattern from AustinLangChain — colinmcnamara · 2026-10-27
- Dev argues there's still alpha in working with your coding agent interactively — alex_frantic · 2026-10-27
- Researchers launch AMA on patterns for safer AI agents used by citizen developers — Loose_Rip359 · 2026-10-27
- Gradio Adds In-Chat Human-in-the-Loop So ML Intern Pauses Before Training — Gradio · 2026-10-27
- Building Halo Infinite Forge maps with local LLMs, DCjson and Blender MCP — swagonflyyyy · 2026-10-27
- Ex-Unity dev Lucas Meijer says OpenClaw looks cleaned up and worth another run — lucasmeijer · 2026-10-27