If No One Can Afford to Train Frontier Models, Can Composition Be Open Source AI's Path?
WebAssemblyMan · reddit · 2026-09-24
The author poses a structural dilemma for open source AI: training a frontier model requires compute and data community developers simply don't have, so the Linux model—where a small kernel grows into a huge ecosystem—may not map onto AI, since the hardest step (training) can't be crowd-sourced.
- Proposed alternative: instead of one open model matching the frontier, build a team of smaller specialists (coding + vision + reasoning + search + tools), combined via composition plus a verifier layer, possibly with human/expert feedback folded in
- Mixture-of-Agents is one version of this; the broader question is whether composition can become the way open AI scales
- Open questions: are there capabilities of a single large frontier model that can't be recreated through composition? What's the equivalent of Linux's "kernel" in this world?
More from AGI Musings
- Perspectives on Machine Consciousness book now available in bookshops and Amazon — cccalum · 2026-09-24
- Computation costs will constrain science research; AI will eventually pose its own questions — miniapeur · 2026-09-24
- Why AI-driven growth requires new companies in every industrial category — chris_j_paxton · 2026-09-24
- Philosopher pokes AP style guide for trying to settle whether LLMs have beliefs — dioscuri · 2026-09-24
- Why Australia isn't prosecuting OpenAI, and the legal gap on AI agent liability — dhadfieldmenell · 2026-09-24
- antirez Tells Young Programmers: AI Now Lets Individuals Build What Once Took Teams — antirez · 2026-09-24