Taylor Approximation Tracks LLM Pre-training Loss
Sham Kakade previews an ICML 2026 report exploring how far Taylor's theorem goes in deep learning, using Lanczos quadrature to calculate the spectral density of a 150M parameter model to track pre-training loss accuracy.
2026-07-09 ~ 2026-07-09 · 2 related posts
- Taylor Approximation Tracks Pre-training Loss — ShamKakade6 · 2026-07-09
- Large Model Spectral Density Analysis to Continue — ShamKakade6 · 2026-07-09