Soumith Chintala: Inkling 2 base model beats GLM-5.3, DeepSeek V4 on code
soumithchintala · x · 2026-11-06
PyTorch creator Soumith Chintala says his team's pre-trained base model is 'extremely competitive,' sharing an internal code benchmark (bits per byte, lower is better): Inkling 2 Small hits 0.2030 pre-trained and 0.2080 post-trained, beating GLM-5.3 (0.2358), DeepSeek V4 Pro Base (0.2335), GLM-5.3 Flash (0.2327), DeepSeek V4.1 Flash (0.2209) and Kimi K3.1 Pro (0.2068). For models with only post-trained checkpoints, they recover base loss via fine-tuning on pretrain-formatted data, validated against models with known base checkpoints.
More from Models
- Claude Opus 5 pulls off multi-level roleplay: staying in character while knowing it's acting — repligate · 2026-11-06
- Inkling 2 Small hits 46.8% Pass@1 on APEX-Agents, up 13 points over predecessor — ziqiao_ma · 2026-11-06
- Inkling 2 Small launches with AA score of 31, weights coming soon — simonguozirui · 2026-11-06
- Inkling 2 Small matches models 8-13x pricier at just $0.34 per task — echen · 2026-11-06
- Dev complains Anthropic free credits can't be used for Code or terminal — RileyRalmuto · 2026-11-06
- Blind listening test of 62 raters ranks Gemini first, Llama last among 8 LLM composers — antiochIst · 2026-11-06