When does a model "know enough" to generalize? A percolation theory perspective

PTenigma · x · 2026-08-24

Core Question: When does a model "know enough" to start generalizing?
Visual Explanation: Every training example is a small ε-ball in the model's embedding space. As more examples are added, these balls clump together into larger groups, "heating up" from green to yellow to red. Generalization begins once they form a giant spanning cluster.
Theoretical Basis: The author uses percolation theory combined with a Lipschitz argument to prove data requirements for training Vision-Language Models (VLMs). Detailed math is available in section §6 of the linked Lecture Notes and the referenced paper "Beyond the Spectral Horizon".

More from Research