iamtrask: AI Risk Is Power Concentration; Antitrust and Copyright as Safety Strategy

On Sept 20, iamtrask, founder of OpenMined and an OpenAI researcher, laid out his core thesis in a tweetstorm: AI risk is best summarized as power concentration. Any risk can only become existential if extreme power concentration sufficient to end humanity already exists; absent accumulated power, AI can hardly cause substantive harm, and RSI is essentially a positive feedback loop of power concentration. The same day he released his Oxford DPhil preprint "Attribution-Based Control in AI Systems," whose core argument is seen as showing that maximizing AI capability runs counter to centralization, echoing the tweets.

Confirmed

• Two policy conclusions: first, a good strategy for minimizing existential risk is to focus on mitigating any form of extreme power concentration—antitrust may thus be the most important AI safety strategy, with the added benefit of accelerating innovation; second, he criticized AI-safety community proposals to further concentrate power (e.g., a world government regulating AI), arguing their implicit premise—that AI power concentration is inevitable or beneficial—does not hold, and that trying to concentrate more power than the risk you are hedging is itself a bad strategy.
• On copyright, he argued the AI safety community should push for hard copyright/data-sovereignty/privacy rulings: short-term they slow AI development, medium-term they naturally align powerful media companies with the safety camp, long-term they decentralize AI power and raise the probability of multiple narrow AIs. He said he repeatedly asked community members why they avoid allying with the pro-copyright camp and never got a decent answer, speculating that a parallel strategy—lobbying AI companies to embrace safety—forces participants into careful status-game positioning. Parallel contradictory strategies are theoretically possible but require someone to forgo salary and prestige to pursue complex litigation, explaining why this route rarely materializes.
• He noted a paradox: antitrust suits explicitly aim to deconcentrate power yet are opposed by the AI safety camp; AI companies themselves want concentrated power, creating a perverse incentive against embracing the "antitrust = decentralization" narrative.
• His operational framework: to reduce AI risk while maximizing benefit, create more competition among narrow AIs while braking general AIs; he also endorsed betterpath.ai's "Pro-Human Path for AI" framework.
• User herbiebradley, in a thought-experiment debate with Trask, countered that realistic copyright rulings would not significantly slow AI training—only accelerate data spending—and that pretraining has passed the point where web-scraped data is largely exhausted, leading him to deem copyright laws restricting training immoral.

Unconfirmed

• herbiebradley's claim that pretraining data is exhausted remains a personal view, with no further resolution of the disagreement with iamtrask's copyright strategy.

Why it matters

• The framework ties the priority of alignment to power structure: alignment matters chiefly when power is concentrated, bridging AI governance and antitrust policy.
• Coming from an OpenAI researcher, it shows decentralization-first thinking has supporters inside mainstream AI labs, relevant to regulatory path choices.
• The simultaneous paper release signals academic grounding beyond social media, while herbiebradley's rebuttal highlights disputed assumptions about copyright's slowing effect.

2026-09-20 ~ 2026-09-20 · 15 related posts

• [source] iamtrask: AI risk is best summarized as concentration of power, not misalignment — iamtrask · 2026-09-20
• iamtrask: It's Topsy-Turvy That the AI Safety Crowd Opposes Antitrust Suits — iamtrask · 2026-09-20
• iamtrask: Antitrust May Be the Most Important AI Safety Strategy — and It Speeds Up Innovation — iamtrask · 2026-09-20
• iamtrask's Framework: Boost Competition for Narrow AIs, Slow Down General AIs — iamtrask · 2026-09-20
• iamtrask endorses betterpath.ai framework: compete on narrow AI, slow down general AI — iamtrask · 2026-09-20
• iamtrask: no existential risk without extreme power concentration, so target that first — iamtrask · 2026-09-20
• iamtrask: Concentrating more power than the AI risk you're mitigating is a bad strategy — iamtrask · 2026-09-20
• iamtrask: AI safety community has no good answer for avoiding the copyright alliance — iamtrask · 2026-09-20
• iamtrask: hard copyright rulings would slow AI, empower media companies on the safety side — iamtrask · 2026-09-20
• iamtrask: AI safety community avoids copyright alliance to protect its vendor-lobbying strategy — iamtrask · 2026-09-20
• iamtrask: parallel contradictory strategies require defecting from prestige to legal fights — iamtrask · 2026-09-20
• [source] Oxford DPhil Thesis by Andrew Trask Argues Maximal AI Capability Runs Against Centralization — iamtrask · 2026-09-20
• [source] Web-scraped pretraining data is tapped out; blocking training on public data is unethical — herbiebradley · 2026-09-20
• Copyright rulings would accelerate licensing shifts, not slow AI training — iamtrask · 2026-09-20

1 near-duplicate retellings: iamtrask