LLM MoEs run at ~5% sparsity, cited as counterexample in consciousness complexity debate
JoshPurtell · x · 2026-10-06
In his debate with Victor Levoso on consciousness and complexity, Josh Purtell points out that LLM MoEs typically run at roughly 5% sparsity due to hardware and training constraints — not because that reflects any fundamental ratio between total and active complexity. He argues this is a hard counterexample to the claim that systems don't carry far more complexity than is actively used.
Related event: How Much Compute Does Consciousness Need? X Users Debate Brain Complexity(9 posts)→
More from AGI Musings
- Dozens of AI agents ran 24/7 with little steering while founder handled admin — kevinnbass · 2026-10-06
- Mollick: 1990s copier repair ethnography shows why AI won't easily replace workers — emollick · 2026-10-06
- Rethinking "Competition Is for Losers": Zero-Sum Skills May Matter More Than Ever in the AI Era — abhiadesai · 2026-10-06
- Consciousness at 10^15-10^16 Ops/Sec? X Users Clash Over Brain Complexity — JoshPurtell · 2026-10-06
- Noah Smith: AI isn't taking college grads' jobs, but it is taking artists' work — i_dg23 · 2026-10-06
- A measurable test for AI consciousness: metamorphic architectures reinvented each token — ryunuck · 2026-10-06