Ripple-65B final run: 12T tokens for 65B params, roughly 9x overtraining
_katieeverett · x · 2026-11-06
Katie Everett 从模型卡补充 Ripple-65B 的最终训练细节:65B 参数模型使用了 12T token(约每参数 185 token,相当于约 9 倍过训练),预训练期间 batch size 从 1408 序列(11.5M token)增至 3072 序列(25.2M token)。她还指出对比图中的点来自 Chinchilla 最优(1 倍过训练)运行,其 FLOP 值与 Olmo 3 阶梯中七个模型尺寸在每参数 20 token 下按 6ND 启发式计算的结果一致。
Related event: Ripple-65B Final Run: 12T Tokens, Roughly 9x Overtraining(2 posts)→
More from Infra
- Melt_Dem: Every Layer of Inference Optimization Will Collapse Into a Market — const_reborn · 2026-11-23
- Canary tokens in ~2M AI requests expose breaches at 3 providers and broken ZDR promises — ccerrato147 · 2026-11-23
- llama.cpp merges token-aware parsing PR for more reliable tool calling and higher t/s — jacek2023 · 2026-11-23
- Inkling 2 Mini: a 26B local model that runs on laptops, and someone rebuilt Apple Notes around it — simonguozirui · 2026-11-23
- Agent-built sandbox platform hits ~1.2k creates/sec, built in a week by 1.5 engineers and 300+ agents — _arohan_ · 2026-11-23
- Context layers don't violate the bitter lesson: LLM-era indexes are data movement economics — CShorten30 · 2026-11-23