LessWrong proposal: burn models into fixed-function ASICs to split AI training from deployment and defuse frontier risk
LessWrong 精选 · rss · 2026-10-29
A LessWrong essay, "Plan R: AI Safety by ASICs," argues today's civilization-scale AI risk stems from Frontier AI Companies combining two properties: (A) creating potentially self-replicating superhuman systems and (B) holding an unbounded financial claim on the resulting surplus. Nuclear/biotech approximate (A) alone and stay mostly safe because nobody profits from pushing safety limits.
The proposal:
- Split entities: heavily restricted AI R&D orgs (no equity, no user data, airgapped, physically enforced multi-hour latency links, write-once logging, run like nuclear labs + G-SIBs) vs. deployment companies (consumer/B2B, can issue equity, move fast).
- Hardware-enforced separation: after evals, models are burned into fixed-function ASICs — circuits physically implement one model, inference only, gradients impossible, so no training and no rogue takeover. Models are then deleted/encrypted from the R&D org.
- Anti-dogfooding: R&D orgs may only use other companies' ASICs, blunting single-lab recursive self-improvement.
- No compute overhang: all high-spec compute outside R&D orgs exists as ASICs, leaving rogue AIs nothing to capture.
AI itself isn't banned — ChatGPT, robots, medicine and military uses continue, just served from ASICs. No entity gets an unbounded claim on the Singularity. Optional hardening: ASICs must prove themselves on real-world tasks for months before use inside an R&D org.
More from Infra
- FlashAttention vs standard attention in one chart: the bottleneck is data movement, not compute — cneuralnetwork · 2026-10-29
- Particle-1 inference goes live on OpenRouter, claiming 28-52% fewer output tokens — ycombinator · 2026-10-29
- Qwen3.8-27B EXL3 quant hits lowest KLD of 0.0002 in just 23 GB — suprjami · 2026-10-29
- MLX-Serve 26.11.1 rewrite: plugin architecture adds models and quants without recompiling — TheMoonMidas · 2026-10-29
- Movistar and Orange activated Starlink coverage after deadly Spain train crash — elonmusk · 2026-10-29
- Microsoft Cloud tops $63B with growth accelerating to 30% YoY as Nadella touts agentic stack — elonmusk · 2026-10-29