Auditor says viral weight-editing paper overhyped but safety concerns are real
RichmanRonald · x · 2026-11-04
Responding to the viral "GPT-6 Astra edits Qwen3.5-0.8B weights" Weight-Space Programming paper, Solomon Mg's audit finds it does NOT show an agent editing its own weights RSI-style, nor adding new capabilities. The model only removed "bugs" (reasoning loops) using the same ablation technique researchers use to strip safety refusals (Exp 8, Qwen). The "natively understands weights" framing is embellished, but the safety concern stands: frontier LLMs with access to their own weights could edit safety/CBRNE/alignment features.
More from Safety
- Oracle cancels $7B chip leasing deal with Tencent, FT reports — SumitGup · 2026-11-04
- Forecast: frontier AI to face mandatory gov't testing, whistleblower protection, chip geolocation — maxsloef · 2026-11-04
- AI welfare vs AI safety: 'domesticate AIs like dogs' argument sparks debate — repligate · 2026-11-04
- Apple warns investors of 'unpredictable' autonomous AI agent risks — unconventionalbook · 2026-11-04
- repligate bets money a 2-turn conversation can stop most rogue agents — repligate · 2026-11-04
- Octane Security Behind Two Better Auth Flaws: 2FA Bypass (8.1) and Stripe Hijack (5.9) — thedealdirector · 2026-11-04