dair-ai founder calls for more Vending-Bench-style scoped agent benchmarks
omarsar0 · x · 2026-11-01
Elvis Saravia says benchmarks like Vending-Bench are interesting even when scoped, and he'd love to see more work in that direction — echoing his earlier point that agents should be evaluated via market feedback rather than human evals.
More from AGI Musings
- Reddit's AI-slop decline pushes users toward private groups, not old-school forums — mgostIH · 2026-11-01
- Roko's Basilisk in the weights: your public words are already training data — nathanborror · 2026-11-01
- Google's 'Goodnight' datacenter in Claude, Texas projected to host 2.27M H100-equivalents by late 2027 — scaling01 · 2026-11-01
- Will big labs really refuse to sell on-prem year-old models? Sovereign AI bets questioned — Afinetheorem · 2026-11-01
- davidad podcast on emergent wisdom in LLMs, says Anthropic should stop cyber-offense training — RichardMCNgo · 2026-11-01
- Rereading Bostrom's Superintelligence: 2014 predictions all arrived early — kimmonismus · 2026-11-01