Grokking observed once or twice during 1B pretraining of opendde-v1, team reports
AllThingsApx · x · 2026-09-13
- itswillhua reports observing model grokking once or twice while training the 1B version of the opendde-v1 official release.
- Notably, this occurred at the pretraining stage, a setting where grokking is rarely discussed at this scale.
More from Research
- World in World: training-free control of frozen video world models for re-camera and revisits — udmrzn · 2026-09-13
- Digital fly with its own connectome brain perceives a real room via AR glasses — stspanho · 2026-09-13
- Jessica Taylor revives 2023 essay: non-superintelligent paperclip maximizers are normal — jessi_cata · 2026-09-13
- Boltz API adds protein fusion, inpainting and sequence redesign, becoming a programmable protein editor — GabriCorso · 2026-09-13
- Sophont + MedARC Claim Lead in fMRI Foundation Models, Eye Structural MRI Next — iScienceLuvr · 2026-09-13
- EvaluatingEvals Grows from 100K to 500K Eval Results Since Beta Launch — evijit · 2026-09-13