DIY coding benchmark: DeepSeek burns 5x tokens yet solves fewer puzzles than Gemini
nankezhishi · reddit · 2026-10-09
A developer, feeling public coding benchmarks mask real capability gaps, built a small custom set of AI-generated programming puzzles to compare Gemini and DeepSeek. Findings:
- DeepSeek did not outperform Gemini in problem-solving
- DeepSeek consumed about 5x the tokens while solving fewer problems
- DeepSeek took roughly twice as long as Gemini
The author says this matches daily dev experience, and notes the final overloaded errors on Gemini/GPT were due to burned-out credits, not failed puzzles.
More from Models
- ChatGPT web can actually listen to audio, but the same model can't via API or Codex — Possible_Statement84 · 2026-10-23
- MiniMax M3.1 Flash free for 3 days on 0G, with 1M context and 512K output — MiniMax_AI · 2026-10-23
- TLA+ expert corrects Grok: TLC only exhausts finite state spaces, not all behaviors — tianyin_xu · 2026-10-23
- OpenAI blocks manual reset with 13% usage left: 'no reset needed right now' — AccomplishedSugar490 · 2026-10-23
- Meta's 816B Muse Spark Open scores 39, losing to 180B Qwen3.8-Flash-Next at 40 — BTA_Labs · 2026-10-23
- Gemini refuses to generate image in Docs, blames document size limit — viksit · 2026-10-23