Open-weight models near 3T parameters as inference emerges as the next moat
ruslansv · x · 2026-07-21
The post argues that open-weight models are already approaching 3T parameters, while a rumored Fable model may have 6–8T parameters. If that trajectory continues, the author expects 20T-scale models by year-end.
The key claim is that inference on such colossal models could become the main moat. The post also notes that training corpora are measured in tens of trillions of tokens, implying that compute-efficient inference may matter more as models and datasets keep scaling.
More from Research
- New review: how prefrontal cortex dynamics turn world models into flexible behavior — AnnaCiaunica · 2026-10-27
- LIBERO-MAX: Robot Policies Drop 11–26 Points When the World Changes Mid-Task — drmapavone · 2026-10-27
- Kolab Benchmark Uses Pro Screen Recordings to Show AI Agents Overestimate Collaboration Quality — minnesotanlp · 2026-10-27
- One BBQ Example Pushes LLM Fairness Scores Past 90%, EMNLP Paper Questions Benchmarks — radamihalcea · 2026-10-27
- Real2Sem2Real turns one phone video into 2,000+ robot demos, matching 150 teleop demos — Scobleizer · 2026-10-27
- Understanding MuonH (Hyperball) through weight norm geometry of scale-invariant networks — jasondeanlee · 2026-10-27