New Agent Benchmark Goes Beyond Vending Bench

xuanalogue's vending-machine agent benchmark differs from Vending Bench by measuring contract violations, not just profit; limited funding prevents broad model coverage, though Sol 5.6 appeared most aggressive and most prone to defaulting.

2026-10-26 ~ 2026-10-26 · 2 related posts

• Vending-bench-style eval: Sol 5.6 appears the most aggressive contract-violator — xuanalogue · 2026-10-26
• New agent benchmark measures model defection rates, not just profit; '5.6 Sol most aggressive' — xuanalogue · 2026-10-26