Code Models Are Insecure by Default; MoE is More Fragile Than Dense Models During SFT
mdancho84 · x · 2026-08-12
The author shares two insights on model architecture and security:
- Code models are insecure by default: Trained on public repos, models inherit a decade of insecure coding patterns. "Safety" fine-tuning doesn't magically fix these code security issues.
- MoE vs Dense = ability vs stability: MoE architectures offer higher capacity but are more fragile during SFT. Hyperparameters matter more, and routing stability becomes a real issue.
Related event: Code Model Security Flaws and MoE Fine-Tuning Fragility(2 posts)→
More from Research
- VIScore: A New Metric for Diagnosing Planning Quality in Latent World Models — DrMorganLevine · 2026-08-13
- Building AI Co-Scientists for the Physical World: Talk at Berkeley Agentic AI Summit — yuqirose · 2026-08-13
- Experiments Show Adam Destroys Low-Rank Bias; Muon Excels at Low Tail Energy — EtherealGlyph · 2026-08-13
- Rethinking AI Energy Metrics: Joules per Token Must Account for Quality and Task — prateekj · 2026-08-13
- Rethinking LLM Generalization: 'General Intelligence' is an Illusion of Aggregated Data — joshua_saxe · 2026-08-13
- Sara Hooker on Gradient-Free Continual Learning and Democratizing AI Compute — AI Engineer · 2026-08-13