Ant International's BizBench: frontier agents often violate business constraints
ant-intl · hf · 2026-10-19
BizBench, from Ant International and Peking University, evaluates long-horizon business agents on intermediate execution validity using real payment procedures. Its executable environment integrates 789 knowledge documents and 601 business tools across 50 multi-turn tasks with resource constraints, temporal dependencies, and conflicting guidance. Frontier models frequently violate constraints even while achieving partial goal success—a costly failure mode in high-stakes domains like payments.
More from coding & agent
- Voice booking agent on ElevenLabs + AWS: tool-turn latency hits 5s P95, how to fix? — matias_flyfishing · 2026-10-19
- Every engineering leader is building the same AI automations right now — generativist · 2026-10-19
- 'Prayer-based programming' emerges as AI coding's latest self-deprecating meme — lxfater · 2026-10-19
- Developer: I keep playing 'human API' in my AI workflows — jacalulu · 2026-10-19
- Vibe coding a parent's TV: Claude Code strips 25 Xiaomi packages, kills boot ads — HarperSCarroll · 2026-10-19
- Sentry co-founder's AI writing workflow: dictate ideas, let an agent outline, write it yourself — lmoroney · 2026-10-19