6 October models, same 5 trap-laden agent tasks: DeepSeek does it all for $0.02, hyped Flint passes 1/5
latent_lurker8587 · reddit · 2026-10-31
A Redditor ran the month's six hottest models through identical small coding tasks (each with a hidden trap like a YAML override or a rename next to a look-alike function) via Atomic Agent 0.6.13 on OpenRouter, 10-minute cap, one run each.
- All five big-name models passed 5/5, but prices diverged 30x: DeepSeek V4.1 Flash finished for $0.02 (median 14s/task), GPT-6.1 Terra for $0.65, Claude Haiku 5.5 was fastest at 12s for $0.035, Gemini 4 Argon and Kimi K3.1 Pro passed at 40s and $0.5.
- Recursive Flint, hyped for a vendor-claimed 74.2 SWE-bench Verified, passed only 1/5 and took 8 minutes on one task — though its free endpoint kept returning 503s all afternoon, so the author cautions the run may reflect a struggling server and invites local benchmarks.
- Keeping the prompt start identical every step meant 79–83% of inputs hit the prompt cache, which is why even pricey models stayed under $1; step-level logs distinguished stuck from slow.
Small sample, a first look rather than a ranking.
More from coding & agent
- Set it on Sunday: a daily prompt that triages Gmail and briefs you via 45-second AI avatar video — toolstelegraph · 2026-10-31
- Next.js prototyping native Go support for Route Handlers and Proxy — rseroter · 2026-10-31
- Tech paradigm shifts reprice mundane infra: 1Password could become the identity layer for agents — signulll · 2026-10-31
- GitHub's Agent Bake Off compares coding models and harnesses head to head — 0xkarasy · 2026-10-31
- Creator builds shipwreck diving game with Opus 5.5 and Astrocade MCP — _amirabs · 2026-10-31
- GitHub Universe talk: 'Build once, run on any agent' with slides and skill files — 0xkarasy · 2026-10-31