Three Clever Uses of Statistical Leverage: Free LOOCV, Robust Stats, and RandNLA Sketches
RobertTLange · x · 2026-11-22
Robert Lange follows up with advanced uses of statistical leverage:
- LOOCV for free: the leave-one-out residual is exactly eᵢ/(1 − hᵢᵢ), so LOOCV = (1/n) Σ (eᵢ/(1 − hᵢᵢ))² with no refitting.
- Robust stats: generalized M-estimators downweight data points with both large residuals and high leverage.
- RandNLA: randomized numerical linear algebra methods use leverage to construct cheap sketches via row subsampling.
A neat tour from classic regression diagnostics to modern randomized algorithms.
More from Research
- François Fleuret asks: is the gradient Japanese? It never refuses to give you a direction — francoisfleuret · 2026-11-22
- Hat Matrix Explained: How Much Leverage Does Each Data Point Have in Least Squares? — RobertTLange · 2026-11-22
- Energy-Based Reasoners: gradient descent on latent states for adaptive reasoning — kalyan_kpl · 2026-11-22
- Lean community certifies Riemann proof; honeypot fake keys expose breaches at three AI providers — LeviTurk · 2026-11-22
- Riemann hypothesis 'proven by Turkish mathematician using AI' claim mocked as the 'Organic Turk' — CatAstro_Piyush · 2026-11-22
- Andriy Burkov releases The Hundred-Page Language Models Book, hands-on with PyTorch — burkov · 2026-11-22