Training quirk: reverse-chronological data ordering worsens loss
kalomaze · x · 2026-10-25
kalomaze shares a training log screenshot: the latest crop validation shows loss 1.4730 at 27.51M tokens, but arranging the data in reverse chronological order degrades it from 1.5124 to 1.5803 — suggesting unusual sensitivity to data ordering.
More from Fun
- Teknium wakes up to Hermes agent Telegram pings while Claude keeps working overnight — Teknium · 2026-10-25
- "Weapons-grade hopium": blogger points to Kurzweil for peak optimism — burny_tech · 2026-10-25
- Simulated video offers a stunning visual tour inside a human cell — CurieuxExplorer · 2026-10-25
- New ChatGPT Instant Has an 'Ackchyually' Problem, Users Complain — gregmushen · 2026-10-25
- Ecco the Dolphin Reimagined with AI Video — demondisc · 2026-10-25
- DP, TP, SP, PP, ZP, FSDP, 1F1B...one engineer lists every training-parallelism acronym — cHHillee · 2026-10-25