Weight edits lift Qwen3.5-9B SecEval from 32.5% to 51.5% without real security capability gains
evilsocket · x · 2026-11-05
- ProjectDiscovery, inspired by the paper "Sparks of Weight-Space Programming," had GPT-6 Astra directly edit the weights of Qwen3.5-9B — no fine-tuning, no gradient descent (targeting layer 23 MLP downproj: W' = W − 4·d·v^T).
- SecEval accuracy jumped from 32.5% to 51.5%, but CTF solve rate only moved 45.7% → 48.8%, code review showed no clear improvement, and vulnerability detection came with 3x more false positives.
- Takeaway: they reproduced the benchmark gain from the original research, but a higher score doesn't mean stronger capability — the model mostly got better at finishing answers it already knew.
More from Safety
- AI 2027 authors 'spooked' as Anthropic S-1 hints recursive self-improvement may be imminent — Puzzleheaded-King584 · 2026-11-05
- Cresta AI Agent earns AIUC-1 certification via independent audit and thousands of adversarial tests — timshi_ai · 2026-11-05
- Google offers $250,000 bounty for a root-level break-in on its own cloud — YvesMulkers · 2026-11-05
- SocialDB scraped creators for fake sponsorship listings, site now down — film_girl · 2026-11-05
- SoftBank cloud arm IDC Frontier hit by ransomware in 'seven minutes,' taking ~500 organizations offline in Japan — ChuckDBrooks · 2026-11-05
- Luiza Jarovsky: AI labs are co-responsible for rising AI cyberattacks and should be legally required to help defend society — LuizaJarovsky · 2026-11-05