Open-Weight AI Security Research Stalls as $80 Fine-Tuning Undoes Safety Interventions
Miles_Brundage · x · 2026-10-28
Miles Brundage (former OpenAI policy VP) amplified concerns that AI security research for open-weight models is stagnating, as researchers neglect the area on overconfident bets that the US and China will eventually ban open weights.
The quoted AI Security Institute blog provides a concrete case: a prior safety intervention wasn't durable — fine-tuning on publicly available viral data fully restored the removed capability with under $80 of compute, even for the largest model. The finding highlights how fragile capability-removal defenses are in open-weight models and how urgent this neglected research area is.
More from Models
- OpenBMB releases open-source MiniCPM-OCR 1.0 with online demo — Dr_Singularity · 2026-10-28
- Liquid AI LFM3's hybrid architecture: 6 global attention layers, 256K context, 25% faster inference — cneuralnetwork · 2026-10-28
- Dev names GLM 5.5 Flash and Qwen 4 27B as the models he's most excited about — victormustar · 2026-10-28
- Maxime Labonne demos d1: bounding boxes for named objects in a single forward pass — maximelabonne · 2026-10-28
- mlx-vlm Will Stay Open Source Under Same License, Only Moving to Org Repo — andrejusb · 2026-10-28
- Users report Grok Bot latency dropped sharply, with noticeably snappier responses — mark_k · 2026-10-28