ProgramBench Eval: Gemini 3.6 Flash Sets New High in Binary Reverse Engineering
jyangballin · x · 2026-08-07
ProgramBench has released new benchmark results for Gemini 3.5 Flash and 3.6 Flash.
- Methodology: The benchmark gives an agent a compiled binary and its docs, asking it to rebuild the entire program from scratch without source code, decompilers, or internet access.
- Results: Gemini 3.6 Flash showed the best performance yet for a Google model in this extreme reverse-engineering task.
Related event: Gemini 3.6 Flash Tops ProgramBench Reverse Engineering Test(3 posts)→
More from coding & agent
- Simon Willison: code mode is the future of tool calling, if it can be sandboxed — springrod · 2026-10-16
- Agent named 'Ilya' reportedly scores 1.1pt higher on benchmarks — daniel_mac8 · 2026-10-16
- Gemini CLI fix: substring matching inlined unrequested binary files into model context — priyanshu1976 · 2026-10-16
- Building AI agents on real healthcare claims data: the domain work that makes them work — alex_verem · 2026-10-16
- OpenAI stealth-testing mysterious internal model 'Aeon' in Codex desktop app and Dots — cedric_chee · 2026-10-16
- Vibe-coded iPhone Duo sniping bot built with Codex in 17 minutes, 41 tests passing — xiaohu · 2026-10-16