Ricursive 65B Model Trained on 12T Tokens With ~9x Overtraining, Power Law Holds
_katieeverett · x · 2026-11-06
Per the model card, Ricursive's final run trained a 65B-parameter model on 12T tokens (185 tokens-per-param, roughly 9x overtraining), raising batch size from 1408 to 3072 sequences (11.5M to 25.2M tokens) during pretraining. Architecture and data changes stack constant-factor improvements while preserving the clean power-law scaling from v1.0.
Related event: Ripple-65B Final Run: 12T Tokens, Roughly 9x Overtraining(2 posts)→
More from Research
- Mathematician proposes making local models great at Lean theorem proving — StefanoGogioso · 2026-11-23
- LLM Study of 1,298 Scenarios Finds Models Blame Men 23 Points More for Identical Misconduct — alex_verem · 2026-11-23
- Far from training data, ReLU classifiers grow more confident with every step — a 2019 theorem explains why softmax can't say 'none of these' — bravo_abad · 2026-11-23
- 55-Page Riemann Hypothesis Proof Guide, De-Formalized from Lean with Gemini and GPT — chaumian · 2026-11-23
- Anders Sandberg on 250,000-page AI proofs and the coming art of proof compression — anderssandberg · 2026-11-23
- Papers on AI timelines are growing super-exponentially, now doubling every 3.8 months — peterwildeford · 2026-11-23