Paper: Why the 'Third Axis' of Exploration is Key to Model Generalization
NielsRogge · x · 2026-08-03
Michael Timothy Bennett's new paper, Why the Third Axis Is Weakness, explores the mechanism of exploration in pre-training. The author argues that traditional single-sample training cannot distinguish between a model that memorizes a single output and one that has learned the entire distribution.
The paper proposes treating exploration as a "third axis" of pre-training and connects it to the concept of "weakness." It proves theoretically that by increasing the number of sampled candidates $K$ and training on the closest match, the probability of meeting all output requirements on unseen contexts is strictly proportional to the model's "weakness." Therefore, when selecting or auditing generative models, one should prioritize measured "weakness" over simple mode counts.
More from Research
- One ancient fold, many jobs: how the immunoglobulin domain evolved into antibodies, MHC and titin — arjunrajlab · 2026-10-16
- Universal forgery attack breaks Chinese PQC signature Origami using only the public key — matthew_d_green · 2026-10-16
- How do you attribute agent gains: model training or infrastructure? — jrxna · 2026-10-16
- First-ever paper on multi-agent swarm safety lands NeurIPS spotlight — LauraRuis · 2026-10-16
- EDEN: anchored embedding noise boosts RLVR exploration beyond policy rollouts — Daijiao Liu · 2026-10-16
- Autonomous vehicle demo shows car reversing to break narrow-road deadlock, with a robotic arm to shift gears — sanjeevs_iitr · 2026-10-16