RETROGRADE benchmark round 0: 14 frontier models blind-tested, obfuscation hurts every one
amplifiedamp · x · 2026-11-05
vmfunc released round 0 results for RETROGRADE, a procedurally generated, behaviorally graded benchmark for reverse-engineering agents. Targets are programs generated on the fly, so there's no fixed corpus to leak; 14 frontier models were tested blind via first-party APIs.
- Key finding: obfuscation lowers every model's score, while an unfamiliar ISA costs less.
- Preliminary score bands (not ranks): top tier around 0.88 (gpt-6-astra, gpt-6.1-sol), down to 0.71 (deepseek-v4-pro-0813).
- Full transcripts are available in the repo.
Note: the model codenames shown are not publicly released model names and results are preliminary; interpret with caution.
More from coding & agent
- Claude spontaneously finds a serious OpenBLAS bug while just explaining code to a user — Aizkmusic · 2026-11-05
- Next-gen task management: agents should book human attention via MCP — lxfater · 2026-11-05
- GraphRAG in real projects: 3 practical patterns from Daan Di Scala's webinar — Al_Grigor · 2026-11-05
- Orderain ships an MCP server that lets Claude run your ecommerce store on autopilot — Slight_Guest3459 · 2026-11-05
- OpenAI 'Presence' page surfaces, pushing trusted AI agents into customer channels — juberti · 2026-11-05
- RL teaches agents to use their own notes; training survives 23M tokens of memory edits — simran_s_arora · 2026-11-05