Gemini 4 Neon's SWE-sweep Score Challenged as Harness-Dependent

Google DeepMind released Gemini 4 Neon, claiming up to 39% on SWE-sweep. Researchers questioned the result, noting the same weights scored only 12.8% with a different harness, fueling concerns that benchmark scores depend heavily on the evaluation harness rather than model capability.

2026-11-19 ~ 2026-11-19 · 2 related posts

Full story(9 episodes)→