ADANA thread: cooldown plus log-time weight decay puts AdamW's equivalent OT on a 1.15 slope
_katieeverett · x · 2026-09-10
Key thread result: log-time memory should in theory reach the loss in O(t) steps that fixed memory needs O(t^1.15) for; with both momentum cooldown and log-time weight decay, AdamW's measured equivalent overtraining roughly follows that 1.15 slope, matching DANA theory.
More from Research
- π0.5 study: raising demos from 10 to 438 lifts Place Ball in Bowl success from 7% to 83% — stingning · 2026-10-27
- HandsOn benchmark: 66 verified tasks and 18,000 human demos for dexterous manipulation — stingning · 2026-10-27
- HandsOn benchmark: 66 human-verified tasks and 18,000 curated demos for dexterous manipulation — stingning · 2026-10-27
- HandsOn splits evaluation: Core-50 execution vs Transfer-16 zero-demo transfer — stingning · 2026-10-27
- Best policy hits only 29.1% on HandsOn; tool use drops to 2.9%, transfer near zero — stingning · 2026-10-27
- FrontierCS 2 launches on Kaggle: 48 tasks make agents beat published CS papers — MeganRisdal · 2026-10-27