DeepSeek's Parallel Attempts Catch Up: pass@2 Beats Luna pass@1 at $0.20
zainhas · x · 2026-08-07
While Luna has higher single-shot quality (67.2% vs 53.3%), DeepSeek's low cost enables parallel attempts: 2 shots reach 81.6% vs 70.1%, 4 shots 90.3% vs 80.5%. For DeepSWE, DeepSeek pass@2 exceeds Luna pass@1 at $0.20 vs $0.61.
Related event: DeepSeek-V4 Flash Benchmarked: One-Sixth Cost, 80% of Luna's Performance(14 posts)→
More from coding & agent
- ERP•AI launches headless SaaS with $0 seats that bills machines: $1 buys 4,000 record writes — rohanpaul_ai · 2026-10-16
- FDE-Gym benchmark shows huge gap on real AI delivery tasks: top model scores 0.563 — DeepLethe · 2026-10-16
- Vibe-coded dashboard leaked franchise financials before anyone logged in — shauntrennery · 2026-10-16
- Fintech HR built a sourcing agent on 60,000 past applicants — 30 qualified candidates in a month — G0NZ0BOY · 2026-10-16
- Orchestrating a 100-image portrait of humanity for aliens with ChatGPT, Argon and p5.brush — dx8xb · 2026-10-16
- 3x DGX Spark cluster runs GLM-5.3-Flash locally behind a proxy, 1.75M tokens/day at 119W — Teknium · 2026-10-16