'Scaling laws are not laws of nature': better data and architectures can still bend the curves
bookwormengr · x · 2026-08-19
The author pushes back against saturation claims: scaling curves can always be bent in a more favorable direction via better curated data, environments & tasks, optimizers, better residuals, embedding size & count, latent attention, latent MoE, and we haven't reached saturation yet.
Core argument: 'Scaling laws are not laws of nature, they are merely statistical extrapolations of the results of the current paradigm' — paradigm improvements themselves can rewrite the so-called laws.
More from Models
- Qwen3.8 Q6 succeeds in local XML/JSON editor coloring task where others failed — megadonkeyx · 2026-08-19
- ChatGPT defaults to French when reasoning level is set low — StephanSturges · 2026-08-19
- Qwen 3.8 27B is missing a 'high' reasoning effort between medium and xhigh — alanoo · 2026-08-19
- Facebook Releases MoE-ViE: Efficient Mixture-of-Experts Vision Encoder — facebook · 2026-08-19
- Qwen3.8-27B users report grammar errors in non-English outputs — LegacyRemaster · 2026-08-19
- DeepSeek Web Chat Wins Industrial Track in TAAC × KDD Cup — jiqizhixin · 2026-08-19