Paper Proposes 'Third Axis' for AI Pretraining: Weakest Hypotheses Maximize Generalization
theomitsa · x · 2026-08-01
A preprint published on Zenodo by Michael Timothy Bennett introduces a new theoretical framework for AI pretraining scaling. Beyond traditional parameters and data, the author identifies a 'third axis'—an exploration count that dictates how many output commitments a generative model can make at once.
This axis is interpreted as 'weakness.' The theory argues that when selecting correct policies that fit the data, choosing the 'weakest' hypothesis (the one compatible with the most further commitments) rather than the shortest one maximizes generalization in inductive reasoning. The paper validates this measurement through controlled experiments across ten image, text, and sequence benchmarks.
More from Research
- DeepMind launches Antigravity for Science: coding agent built for end-to-end research workflows — pushmeet · 2026-10-16
- It's not gradient descent: burny_tech corrects the diagnosis to RLHF mode collapse — burny_tech · 2026-10-16
- A beginner-friendly introduction to causal inference kicks off, Part 1 — Nasereliver · 2026-10-16
- FBI method trains for Pass@K with one sample per update at 12.3% of the cost — EmtiyazKhan · 2026-10-16
- Berkeley researchers sample humanoid robot policies directly from weight space with diffusion — berkeley_ai · 2026-10-16
- OneWM: unified causal training shows world and action modeling improve each other — IAAI233 · 2026-10-16