Deep Nets Induce Programs From Data, Not Simplicity — Debate Contrasts Them With Solomonoff Reasoning
jd_pressman · x · 2026-10-10
In an X thread, jdpressman argues that deep networks don't weigh hypotheses by simplicity the way a Solomonoff reasoner would — and neither do humans. His take: nets start from a library of programs closely based on input data, then generalize by fitting programs that reproduce target data across more inputs while keeping program size fixed — inference in the opposite order to Solomonoff.
- A reply notes Solomonoff can allow short code that runs arbitrarily long, while deep nets have bounded runtime.
- He cites the arXiv paper "Alien Coding" (Gauthier, Olšák, Urban, 2023): a self-learning program-synthesis loop that alternates training a neural machine translator on sequence-program pairs with proposing new programs, discovering programs for over 78,000 OEIS sequences, sometimes inventing unusual programming methods.
More from AGI Musings
- Anil Seth's 'Conscious AI and biological naturalism' collection now open access — anilkseth · 2026-10-10
- AI is laundering human work and knowledge, argues call for idea provenance tracking — asusarla · 2026-10-10
- Spend two years at a Fortune 500 before claiming AI will replace all human labor — burny_tech · 2026-10-10
- Satire about "outraged mathematicians" shows some people actually believe it — burny_tech · 2026-10-10
- AI Is Restructuring Society's Value Hierarchies — Single-Objective Players Become NPCs — GhaffariMaani · 2026-10-10
- Is It Fine to Say AI Has No Consciousness? A Debate Over Whether We're Asking the Wrong Question — asusarla · 2026-10-10