Hessian matrix primer links second derivatives to neural-network optimization
burny_tech · x · 2026-07-21
- The post explains the Hessian matrix, defined as the square matrix of second partial derivatives. - In the 2D case, it reduces to the familiar matrix containing \(f_{xx}\), \(f_{xy}\), \(f_{yx}\), and \(f_{yy}\). - The image shows Otto Hesse, visual examples of curved surfaces, and the compact notation for Hessians. - It also notes that in neural network training, the Hessian can support second-order optimization methods that refine parameters more efficiently than plain gradient descent.
More from Models
- OpenCodex turns OpenAI’s Codex harness into a multi-provider coding workflow — arrakis_ai · 2026-07-21
- A user says Claude 4.6 felt worse yesterday and asks whether model quality can drift over time — Rahios · 2026-07-21
- Researchers debate whether GPT-OSS ever had a clear harm case — aiamblichus · 2026-07-21
- Kimi K3 hits 89.4% peak on software tasks while Fable 5 is slightly steadier — FinanceYF5 · 2026-07-21
- Kimi K3 leads on Go, but Fable 5 wins Python, JavaScript, TypeScript and Rust — FinanceYF5 · 2026-07-21
- Kimi K3 reaches 89.4% pass@4 and tops the benchmark over GPT-5.6 Sol — FinanceYF5 · 2026-07-21