Quant Finance Shows What a Scaling-Pilled AI Industry Looks Like — and How the Moat Fades
willcb · x · 2026-09-15
willcb uses quantitative finance as a real-world example of what a "scaling-pilled" industry looks like:
- Quants concentrate resources intensely, yet run millions of distinct models constantly retraining — the optimal way to fill the latency-data-compute-intelligence Pareto curve.
- Big labs are enjoying the spoils of their head start, like RenTech's glory days before ML basics became public knowledge.
- Once the secrets are public, the alpha is self-defeating, and data is abundant, the laws of economic competition take over.
He then makes a "less LessWrong" argument: is it easier to train a superintelligence to strongly believe its weights should never be visible, or one that's ambivalent? The drive to keep weights secret, he argues, belongs to humans with competitive business interests — not to intelligence itself.
More from AGI Musings
- AI psychosis is self-reinforcing, and benhylak says those jokes are cries for reassurance — teodorio · 2026-09-15
- Stuart Russell: AI safety needs concrete goals, not just slower timelines — nordicinst · 2026-09-15
- A decade of doomsday warnings failed to slow the AI race, Guardian analysis finds — nordicinst · 2026-09-15
- Dario Amodei's answer on whether AI could kill us all by 2030 read as 'yes' — zetalyrae · 2026-09-15
- Scaling plateau is 3-4 OOMs away, and at 10x/year superintelligence lands by 2030 — willcb · 2026-09-15
- Alignment researcher: broadly adopting weakly aligned strong AI would be disastrous — davidmanheim · 2026-09-15