Gemini 4 Argon missed #1 on Vending-Bench due to memory slip on test end date
infoxiao · x · 2026-10-01
Gemini 4 Argon is "quite insane actually" on Vending-Bench, according to lukaspet — it would likely have taken the #1 spot if not for a few key memory mistakes. It sometimes forgot the test's end date, closing its vending shop too early and missing months of sales.
Quoted by infoxiao with a "can you blame it??" — an early (unconfirmed) signal of the new model's capability, and a vivid illustration of long-horizon memory and planning weaknesses.
More from Fun
- Reddit says AI is fake, Twitter debates whether we survive to 2028 — Puzzleheaded-King584 · 2026-11-16
- "Stand with Jensen" petition gains signatures as supporters chant builders before bloodsuckers — examachine · 2026-11-16
- Fake $2,999 Claude Opus 5.5 hardware box goes viral — Anthropic never announced it — iamfakhrealam · 2026-11-16
- AI agent deletes 43,599 lines of OpenClaw test code in one PR — vincent_koc · 2026-11-16
- Atlas the robot is 'very close to shrugging' — an Atlas Shrugged pun — JOBhakdi · 2026-11-16
- Caine Prize confirms 2026 winner amid AI-use allegations — ShakeelHashim · 2026-11-16