Neural Nets Generalize by Fitting Data-Suggested Programs, Not the Simplest Ones
jd_pressman · x · 2026-10-10
jdpressman speculates that deep nets generalize by starting from a library of programs closely based on input data, then fitting programs that reproduce/predict target data across more and more inputs while keeping program size the same. Like other successful program-search approaches, neural nets are data-driven: they find programs suggested by features of the data, not the simplest or most general ones — inference in the opposite order to a Solomonoff reasoner. He links the arXiv paper "Alien Coding" as context.
More from Research
- Anil Seth's 'Conscious AI and biological naturalism' collection now open access — anilkseth · 2026-10-10
- Inner Logic's Axel Krieger: surgery is a perfect fit for imitation learning — ___Mufasaa · 2026-10-10
- Benchmark fatigue: a new hard AI benchmark doesn't have to measure performance at all — adonis_singh · 2026-10-10
- Hidden two-file test: 3 of 8 agent fixes closed the reported bug by breaking another — lulzxdxdxd · 2026-10-10
- Researcher spotlights paper: well-engineered B-Trees handle in-memory and out-of-memory workloads — arpit_bhayani · 2026-10-10
- Mathematicians pinpoint the moment AI cracked the Exact Overlaps Conjecture via Astra transcript — vishalmisra · 2026-10-10