Using Curvature to Measure True Representation Convergence
its_vayishu · x · 2026-07-18
The author points out that when measuring how much a representation "moves" across layers, arc length is a poor metric because it merely accumulates step sizes: even if the trajectory bounces back and forth, the arc length will be large, but it doesn't signify actual progress.
He advocates for using curvature—the angle between consecutive displacement vectors—to determine if the direction is stabilizing.
- Low curvature: Indicates that the representation in the later layers of the network is settling on a direction and continuously advancing.
- Distinct from distance/arc length: Curvature differentiates between "true convergence" and "just oscillating motion."
- This is particularly useful in layer-wise interpolation / latent trajectories, and it is very cheap to compute.
More from Research
- Causal-only attention for non-generative tasks is wasteful, argues HF engineer — antoine_chaffin · 2026-09-11
- Catholic University of Chile researcher: scaling AI feedback is key to sustainable medical education — julianvarascom · 2026-09-11
- Nature paper images cellular activity across all organs, revealing body-wide circuits — arjunrajlab · 2026-09-11
- SignNet 1M Dataset Released for Sign Language Research — ducha_aiki · 2026-09-11
- ECCV26 Oral: Flow Matching Enables Single-Stage Multi-View Point Cloud Registration — ducha_aiki · 2026-09-11
- InFlux++ Method Released — ducha_aiki · 2026-09-11