Using Curvature to Measure True Representation Convergence
its_vayishu · x · 2026-07-18
The author points out that when measuring how much a representation "moves" across layers, arc length is a poor metric because it merely accumulates step sizes: even if the trajectory bounces back and forth, the arc length will be large, but it doesn't signify actual progress.
He advocates for using curvature—the angle between consecutive displacement vectors—to determine if the direction is stabilizing.
- Low curvature: Indicates that the representation in the later layers of the network is settling on a direction and continuously advancing.
- Distinct from distance/arc length: Curvature differentiates between "true convergence" and "just oscillating motion."
- This is particularly useful in layer-wise interpolation / latent trajectories, and it is very cheap to compute.
More from Research
- New review: how prefrontal cortex dynamics turn world models into flexible behavior — AnnaCiaunica · 2026-10-27
- LIBERO-MAX: Robot Policies Drop 11–26 Points When the World Changes Mid-Task — drmapavone · 2026-10-27
- Kolab Benchmark Uses Pro Screen Recordings to Show AI Agents Overestimate Collaboration Quality — minnesotanlp · 2026-10-27
- One BBQ Example Pushes LLM Fairness Scores Past 90%, EMNLP Paper Questions Benchmarks — radamihalcea · 2026-10-27
- Real2Sem2Real turns one phone video into 2,000+ robot demos, matching 150 teleop demos — Scobleizer · 2026-10-27
- Understanding MuonH (Hyperball) through weight norm geometry of scale-invariant networks — jasondeanlee · 2026-10-27