Marin's 535B pretraining passes 80%, loss runs below scaling-ladder forecasts
dlwh · x · 2026-10-24
Marin's 535B model (Hero 535B-A23B) is now over 80% through pretraining. Observed held-out loss is tracking below the scaling-ladder forecasts; a sharp drop coincides with the planned 4K→16K context switch, which the team admits makes the comparison somewhat unfair, so checkpoints will be re-evaluated at 4K. Loss currently stands at 1.528 at step 314,999.
More from Research
- Ex-OpenAI's Yaroslav Bulatov on why backprop should go: it's the memory wall — yaroslavvb · 2026-10-24
- UBC NLP lands 21 papers at EMNLP 2026 spanning culture, RAG, and agents — mageed · 2026-10-24
- Terminal-Bench exposes NVIDIA Nemotron 3.5 near-zero score vs Kimi K3.1's 65.7% — JJitsev · 2026-10-24
- Kimi K3.1 Pro reportedly first Chinese model to solve 20x20 nonograms — fairydreaming · 2026-10-24
- Self-Evolving Benchmarks: Multi-Agent Evolutionary Search Auto-Generates Challenging Tool-Use Tasks — MichaelD1729 · 2026-10-24
- Colossal didn't de-extinct the dire wolf — they gene-edited gray wolves to look like them — shae_mcl · 2026-10-24