Analyst: new open-source 2S model is clearly undertrained, room for 3x more tokens
teortaxesTex · x · 2026-11-07
X user teortaxesTex offers a detailed take on a new open-source model (2S):
- It is clearly undertrained — at current budgets it could handle at least 3x more tokens (MiMo Flash did 4x); post-training could scale an order of magnitude via MOPD + MixRL, with more environments and data all trending upward.
- On this trajectory, he predicts the upcoming 2T model, if released within 3 months, will be around 6.1 Sol level or certainly above 6.1 Terra — while cautioning that continued scaling needs adjusted hyperparameters and data mixes or returns may diminish.
- Verdict: heads and shoulders above US open source so far, on par with the best Chinese efforts but with a steeper trajectory.
- Addendum: he is genuinely worried for the "Whale" — its infra-driven designs may be suboptimal and non-scalable due to a weak scaling ladder and possibly weak data, and it may fail to convincingly trounce this model.
Related event: Chinese open-source models stun benchmarks, leaving Meta behind(3 posts)→
More from AGI Musings
- Agent-native email will replace Gmail as the connective tissue for agentic commerce — signulll · 2026-11-07
- Anthropic study: AI could absorb 27-52% of researchers' time, median 38% across fields — emollick · 2026-11-07
- Nearly 30% of ICLR 2027 reviews mention "appendix", fueling AI-written review fears — nsaphra · 2026-11-07
- "A year or two from now you'll just tell Astra 9 to make your humanoid do it," says SF builder — carlosdponx · 2026-11-07
- Stanford's Anshul Kundaje: ML conference peer review is broken and unfixable — anshulkundaje · 2026-11-07
- Eric Drexler: AI options, not optimism — participants weigh options, not odds — sebkrier · 2026-11-07