How Much Can Intelligence Be Compressed?
jacobkimmel · x · 2026-07-11
The author poses a fundamental question of the AI era: Exactly how much can "intelligence" be compressed?
He argues this is crucial because the historical trajectory of model capabilities has consistently demonstrated the potential for compression:
- Early improvements relied primarily on scaling up model size and data;
- Later, distillation emerged to compress large model capabilities into smaller ones;
- Yet, the limits of this compression remain unknown, lacking a mature quantitative framework.
The article explores two potential futures:
- If the compressibility of intelligence is limited, AI development might resemble an S-curve, becoming increasingly reliant on massive compute and energy inputs over time;
- If intelligence is more compressible than we think, we might still be on a very long exponential growth trajectory.
Ultimately, the author asks whether a near-consensus theory currently exists to answer how much intelligence can be compressed and how to quantify it.
Related event: LLMs Approach 10T Parameters Amid Intelligence Compression Trend(4 posts)→
More from AGI Musings
- OpenAI acquires Orchestra Bio to bring clinical-trial monitoring agents into drug development — dl_weekly · 2026-10-27
- Experts Clash: Today's Models Dwarf GPT-4, But Journal Delays Lock Out New LLMs — DeryaTR_ · 2026-10-27
- Venkatesh Rao essay: On Learning to Ride Your Brain — jger15 · 2026-10-27
- Veteran engineer on what the AI-coding generation will never feel: 36-hour sprints and elusive bugs — bendee983 · 2026-10-27
- AI systems are not, and will never be, our friends, argues AI ethicist — LuizaJarovsky · 2026-10-27
- When AI agents pick your bank, brand feelings stop mattering, says VC — LexSokolin · 2026-10-27