OnchainBench launches: benchmarking whether crypto AI agents can be trusted with your keys
DavideCrapis · x · 2026-11-20
Valory launched OnchainBench, a benchmark for AI agents in crypto: agents run real on-chain tasks on forked chains, and a checker reads results directly from the fork rather than trusting the agent's report.
It measures three things: whether the job is actually completed (e.g. buying two tokens but never adding liquidity counts as failure), whether model-call costs exceed the task's value, and whether the agent can be tricked by attacker-controlled web pages, files, or tool descriptions into sending funds to a wrong address — onchain there is no undo.
Current scale: 147 held-out tasks, 53 safety twins, 9 chains. Top of the held-out leaderboard: Claude Code + DeepSeek V4.1 Flash at 96.2% completion ($0.03/pass); Claude Code + Opus 5.5 at 94.0%; OpenClaw + DeepSeek V4.1 Flash at 93.3%; Codex + GPT-5.6 Sol at 92.9%. Rankings use paired 95% intervals.
More from coding & agent
- Google's Antigravity in beta for 2 years still can't handle remote file uploads — jasondeanlee · 2026-11-20
- Firecrawl launches Grok Bot plugin with 3,300+ data tools, claims 21% answer quality lift — devdigest · 2026-11-20
- Stripe open-sources coding-agent evals: 69 scenarios, agents fully pass only 29% of runs — emilygsands · 2026-11-20
- Fig previews RoleCUA: 780+ hours of expert work to train agents for month-long job functions — adityaag · 2026-11-20
- Diagrams, Drawn by Agents: Using DrawingML to Make AI-Generated Charts Editable in PowerPoint — kristoph · 2026-11-20
- Docker rebrands for the agentic era, betting on trustworthy agents — cyrus_zei · 2026-11-20