kalomaze shares loss curve of larger-backbone run on DeepSeek V3.1 base, eval loss down to ~0.82
kalomaze · x · 2026-11-04
kalomaze posted a training curve screenshot illustrating what he means by a "larger backbone": a finetune of deepseekv31base with fp8, r16 mixing, WSD + ADEMA settings on 16h200 hardware. Eval loss falls steadily from 1.14 to 0.82 over roughly 3,750 steps. This is a thread fragment; details depend on the preceding tweets.
More from Research
- Cambridge's van der Schaar Lab opens PhD applications for 2027 entry — MihaelaVDS · 2026-11-04
- Tencent's Hyra 2.0 Generates Protein Sequences 57x Faster Than ProteinMPNN — YvesMulkers · 2026-11-04
- Researcher Urges Top AI Conferences to Desk-Reject AI-Generated Papers Amid Fake Data Scandals — Vegetable-Formal-753 · 2026-11-04
- Harvard Professors Admit AI Article Reviews Now Beat the Best Humans — iamfakhrealam · 2026-11-04
- Pre-pretraining gains traced to embedding init: N(0,0.02) alone lifts perplexity 9% — LauraRuis · 2026-11-04
- Hobbyist rewrites Qwen 0.6B weights with 39.8M tokens, no gradients, in 3h12m — Altruistic_Heat_9531 · 2026-11-04