SII-GAIR runs RSI on a full frontier-scale pre-training: 30B/A3B with 134B agentic tokens
AdinaYakup · x · 2026-11-05
- SII-GAIR and Xinxu AI shared detailed insights from a rare experiment: applying RSI to an entire frontier-scale pre-training run.
- Key numbers: 30B total parameters / 3B active (MoE); 134B agentic tokens included in pre-training.
- Core method: AI agents improve pre-training through debugging, evaluation, and feedback loops.
- Coming soon: base model weights, a tiny SFT dataset (already updated on Hugging Face, 271k downloads), and a detailed technical report.
More from Models
- GPT-6 Cyber Allegedly Priced at $20/$100 per Million Tokens, Leaks Suggest — imjustnewatai · 2026-11-06
- Rumor: GPT-6.2 Sol and Astra deployed on Google Cloud, release imminent — soumitrashukla9 · 2026-11-06
- Leaked Screenshot Shows Alleged GPT-6 Cyber Scoring 48.5% on Benchmark — scaling01 · 2026-11-06
- OpenAI launches GPT-6 Cyber security model already used by Mozilla, Cisco and Palo Alto Networks — OpenAI · 2026-11-06
- OpenRouter spending chart shows open models repeatedly breaking into top five — Scobleizer · 2026-11-06
- DeepSeek's staff mostly work on data; Liang Wenfeng labels data himself, LatePost reports — teortaxesTex · 2026-11-06