MalwareBench: first end-to-end malware RE benchmark, best model scores just 4.42%
brianryhuang · x · 2026-10-23
The genanalysis team released MalwareBench, billed as the first end-to-end, cross-platform benchmark for malware reverse engineering. Unlike saturated RE benchmarks, models perform poorly here: GPT-6.1 Sol leads with only 4.42%.
A typical run takes several hours and hundreds of millions of tokens, highlighting both the task's difficulty and the inference cost of evaluating it. A useful signal for gauging frontier models' real capability in security-analysis workloads.
More from Models
- Drex Max shares vision benchmarks: 74.5 Public Vision Index vs GPT-6 Luna's 63.9 — nischay_twt · 2026-10-23
- Drex Max debuts: heavily distilled decision model claims top-3 on public benchmarks at $0.06/M tokens — nischay_twt · 2026-10-23
- Dev who shilled GPT for a year says Claude Opus 5.5 is 'so nice to work with' — rudrank · 2026-10-23
- DeepSeek V4.1 Flash matches Gemini 3.1 Pro on ARC-AGI-2 at ~1/6 the cost — teortaxesTex · 2026-10-23
- Smaug to integrate Nvidia Nemotron: 'run a powerful LLM from one machine' teased — bindureddy · 2026-10-23
- Claim: Opus 5.5 quietly nerfed to make the next model look like a bigger jump — RileyRalmuto · 2026-10-23