V4-Flash vs Luna: Contrasting Performance on SWE Benchmarks
teortaxesTex · x · 2026-08-09
A technical discussion compares the performance of V4-Flash and Luna models on the SWE benchmark. Data shows V4-Flash pass@2 outperforms Luna pass@1, but narrowly falls behind Luna pass@2 at pass@4. This contradicts the intuition that Luna is more RL-fried, suggesting Luna might simply be a larger model with greater diversity and more knowledge in SWE.
Related event: Anonymous Luna Model Shows Strong SWE Benchmark Performance(2 posts)→
More from Models
- willcb: Opus 5.5 doesn't smell bigger, it smells like they fixed the data — willcb · 2026-10-16
- Unverified claim: Apple and Meta's 816B-parameter Muse Spark Open runs on two Mac Studios — TansuYegen · 2026-10-16
- Gemini 4 model card: model eliminated months of trial-and-error on bioweapons research — Puzzleheaded-King584 · 2026-10-16
- Every AI picks 7: Anthropic engineer's rule for escaping model taste convergence — rubenhassid · 2026-10-16
- Brain teaser: strip all attention, MLP and embedding matrices from a Llama 3 — how many trainable params remain? — Sauers_ · 2026-10-16
- Lean v4.35.0 ships verified kernels — one written and proven consistent by Claude — geoffreyirving · 2026-10-16