Hat Matrix Explained: How Much Leverage Does Each Data Point Have in Least Squares?
RobertTLange · x · 2026-11-22
Robert Lange shares a concise three-step explainer on the hat matrix and statistical leverage in least-squares regression:
- The fit is linear in y: ŷ = Hy with H = X(XᵀX)⁻¹Xᵀ — H 'puts the hat on y'.
- Diagonal entries hᵢᵢ are leverage: hᵢᵢ = ∂ŷᵢ/∂yᵢ, so nudging yᵢ by Δy moves its own fitted value by hᵢᵢ·Δy.
- Leverage depends only on x (not y) and sums to p (number of parameters), averaging p/n — diluting any single point's influence as n grows.
A useful refresher on the fundamental tool for regression diagnostics and measuring data-point influence.
More from Research
- François Fleuret asks: is the gradient Japanese? It never refuses to give you a direction — francoisfleuret · 2026-11-22
- Energy-Based Reasoners: gradient descent on latent states for adaptive reasoning — kalyan_kpl · 2026-11-22
- Lean community certifies Riemann proof; honeypot fake keys expose breaches at three AI providers — LeviTurk · 2026-11-22
- Riemann hypothesis 'proven by Turkish mathematician using AI' claim mocked as the 'Organic Turk' — CatAstro_Piyush · 2026-11-22
- Andriy Burkov releases The Hundred-Page Language Models Book, hands-on with PyTorch — burkov · 2026-11-22
- Generative AI meets cardiology: Dagstuhl seminar bridges ML and clinical practice — StephanMandt · 2026-11-22