Training-Physics Intuition Compressed Into a Decision Tree Hits 78.2% Out-of-Fold
ZimingLiu11 · x · 2026-10-24
Part 3 of the thread: LLMs draw on "training physics" heuristics—cumulative effective learning rate U (Adam ≈ ηT, Adagrad ≈ 2η√T), NTK curvature, and zero-momentum SGD failures. Compressing this intuition into a human-readable decision-tree policy achieves 78.2% out-of-fold accuracy, showing intuition can be explicitly distilled into rules.
More from Research
- Why KV sharing works: 60-layer model keeps 21% of KV cache with lower loss — yoavartzi · 2026-10-24
- Researchers warn Opus 5.5 pushes back with silly analysis, risking false negatives — adam_karvonen · 2026-10-24
- Terence Tao: AI turns beautiful mathematics into 'toxic waste' — TheObserverUK · 2026-10-24
- Matt Niessner's lab retreat: CVPR paper reviews and newly discovered dancing — angelaqdai · 2026-10-24
- Mistral's Timestral tops time-series benchmarks with 209M params, built mostly by agents — hkproj · 2026-10-24
- Claim: task-free continual learning in O(1) memory with zero replay and zero backprop — Horror-Celery4869 · 2026-10-24