Countervailing Technologies: how distillation-style tools can diffuse AI power concentration
DavideCrapis · x · 2026-10-05
Oxford researcher Samuele Marro's essay "Countervailing Technologies" argues AI is an engine of power concentration, creating gap-driven risks alongside better-known access-driven ones. Standard fixes (pauses, treaties, benefit-sharing) are coordination games vulnerable to bootstrapping problems and rational refusal by the powerful. He proposes countervailing technologies—power-diffusing, unilaterally adoptable, veto-resistant, cheap to replicate—with the crossbow and model distillation as examples. He also covers second-order effects (more access risk, incumbent backlash, re-concentration) and lists directions for building them.
More from AGI Musings
- Bengio cites poll: 73% of Americans fear AI could threaten human survival — Yoshua_Bengio · 2026-10-05
- Quanta: Is AI the End of Math As We Know It? — littmath · 2026-10-05
- An AI world without vocations would demoralize creators, essay argues — zetalyrae · 2026-10-05
- Eric Topol highlights major RAS pancreatic cancer advance, MYC next in line — EricTopol · 2026-10-05
- Anders Sandberg's best debate trick: back-to-back votes to find real disagreement — anderssandberg · 2026-10-05
- Karpathy: stop sending slides — generate throwaway pages and videos to explain things — sujingshen · 2026-10-05