26-30B is the best dense model size range, argues ML commentator
auto_grad_ · x · 2026-10-20
The author argues that 26-30B parameters is the best size range for dense models, with nothing else coming close, and expresses respect for teams scaling capabilities at this size. This reflects a "small but strong" school of thought: mid-sized dense models remain favored for deployability and cost-efficiency amid the dominance of very large and MoE architectures.
More from AGI Musings
- François Fleuret: all AI efforts should go to making AI love and understand us — francoisfleuret · 2026-10-20
- Pedro Domingos jokes we're nowhere near AGI — 'Artificial Goat Intelligence' — pmddomingos · 2026-10-20
- Penrose: Gödel's theorem sets a hard limit on what AI can do — JosephJacks_ · 2026-10-20
- Domingos: AI regulation 'creates real problems by fixing imaginary ones' — pmddomingos · 2026-10-20
- Artist mentor of 8 years laid off as company keeps only one AI-savvy artist — Fresh-Personality598 · 2026-10-20
- Pedro Domingos: The 'society of models' I predicted in 2015 is coming true — pmddomingos · 2026-10-20