View: Pretraining Far From Saturated; Scale Not Necessary for Peak Intelligence
kalomaze · x · 2026-08-21
Twitter user kalomaze discussed the sources of model intelligence in a reply:
- Pretraining Potential: The latent capabilities within the oldest dsv3 (DeepSeek-V3) pretraining checkpoints are far from saturated. One could freeze pretraining for 5 years, and the outcomes from unlocking that potential would still be unrecognizable.
- Model Scale: Referencing the 0731 ARC-AGI-2 score, the 0813 performance, and professor's views, model scale is argued not to be a necessary condition for peak intelligence.
- Post-training Space: While stronger pretrains are valuable for raw 'g', the space of tractable process goals (pg objectives) is incomprehensibly wide. Current efforts are only tapping the lowest-hanging fruits like rubrics and RLVR.
More from AGI Musings
- Claude Shannon's 'The Bandwagon': A warning against hype from the info age — vsikka · 2026-08-21
- Exercising the Idea Muscle: Creativity in the Age of AI Commoditization — dejavucoder · 2026-08-21
- Opinion: Programmers Directing AI Coding Might Be a Detour — gefei55 · 2026-08-21
- Musk Predicts AI and Robotics Could Increase Global Economy 10-100x — MickeySteamboat · 2026-08-21
- From answering questions to managing work: what should proactive agents notice? — greatlearningglobal · 2026-08-21
- Parag: 1000x agent web use is a huge underestimate — schwentker · 2026-08-21