Open-weight models near 3T parameters as inference emerges as the next moat
ruslansv · x · 2026-07-21
The post argues that open-weight models are already approaching 3T parameters, while a rumored Fable model may have 6–8T parameters. If that trajectory continues, the author expects 20T-scale models by year-end.
The key claim is that inference on such colossal models could become the main moat. The post also notes that training corpora are measured in tens of trillions of tokens, implying that compute-efficient inference may matter more as models and datasets keep scaling.
More from Research
- Stanford Team Introduces Gigatoken, the World's Fastest Tokenizer — StanfordAILab · 2026-07-22
- Tabul AI launches Metal TreeSHAP to speed up Shapley values on Apple silicon — Scobleizer · 2026-07-22
- Reddit points to OpenAI’s ChatGPT Ads page — EcstaticAsparagus509 · 2026-07-22
- Open-source runtime lets each repo define its own AI code reviewer — ibabufrik · 2026-07-22
- DeepSWE: A New Benchmark for Evaluating AI Coding Agents on Real GitHub Issues — pmz · 2026-07-22
- A Rust space-economy sim runs hundreds of autonomous ships, built with Claude — kalcode · 2026-07-22