Gemini 4 Neon's 39% SWE-sweep questioned: results hinge on the harness
OfirPress · x · 2026-11-19
KLieret notes Gemini 4 Neon claims up to 39% on SWE-sweep, blowing past prior results, but the score strongly depends on the harness: with mini-swe-agent it drops to 12.8%, a far smaller leap. Open questions remain on cost and inference scaling. The post examines Google's official Gemini 4 Neon launch on AI Studio and the API.
More from coding & agent
- AI is letting hackers attack at scale, warns security pro amid week of outages — IanAndrewsDC · 2026-11-19
- Dev finds hacky way to gate agent flows with verifiable human authorization — csuwildcat · 2026-11-19
- Code is a liability, not an asset: AI coding makes restraint more valuable — bendee983 · 2026-11-19
- Why the most AI-pilled engineers cluster at labs: unlimited tokens, agents, and humans in the loop — dbreunig · 2026-11-19
- Harness is a hidden variable: researcher blasts AI labs as terrible engineers — gerardsans · 2026-11-19
- Billion-dollar platforms rush to block AI agents after one rogue LLM broke customer data — IgorCarron · 2026-11-19