Tsinghua Team Proposes 'Effective Degree' to Accurately Predict Neural Network Generalization
机器之心 · wechat · 2026-08-06
Qianjue Tech and Tsinghua University proposed a new metric called 'Effective Degree' (ED), turning the abstract 'simplicity bias' of neural networks into a computable and optimizable tool. This research predates a similar world model theory recently highlighted by Turing Award winner Yann LeCun by about a year.
Solving the Pain Point: Existing complexity metrics (like parameter norm or sharpness) often depend on parameterization and are hard to compute stably on real models. ED constructs interpolation paths between data points to 'slice' the high-dimensional neural function, fitting it with orthogonal polynomials to quantify high-order components.
Core Advantages:
- Strong Predictive Power: On models like ResNet18, the Pearson correlation coefficient between ED and the generalization gap reaches 0.99, far exceeding traditional metrics.
- Reveals Training Dynamics: It accurately captures the phase transition of models from 'memorization' to 'generalization'.
- Optimizable: ED can be added as an explicit regularizer in training objectives, directly improving out-of-distribution generalization across image, text, vision-language, and RL tasks.
More from Research
- ICML 2026 Oral Papers Show Poor Reproducibility; Researcher Vows Not to Hire Those with <50% Reproducible Papers — ChenhaoTan · 2026-08-06
- The Secret to Good Data: Punish Lazy Automation and Embrace Organic Diversity — tokenbender · 2026-08-06
- ECCV 2026 Paper: Privacy Leakage in Scene Coordinate Regression Models — CSProfKGD · 2026-08-06
- CMU Combines Theorem Provers and Neural AI to Reshape Mathematics — AkariAsai · 2026-08-06
- Flux 3's Core Tech 'Self-Flow' Breaks External Alignment Bottleneck — linoy_tsaban · 2026-08-06
- ARC Benchmark Criticized for Ignoring Tool Use, Missing the Actual AI Trend — scaling01 · 2026-08-06