Researcher Speculates: LLMs Explore Hypothesis Space in Near-Solomonoff Order
jd_pressman · x · 2026-10-10
A discussion thread on how deep nets versus human mathematicians search hypothesis space. The author argues that even if deep nets start with complex hypotheses and whittle them down, human mathematicians — and LLMs doing math and physics — clearly explore hypotheses in something closer to Solomonoff order (simplest programs first). He further guesses the mechanism: models start from a library of programs closely based on input data, then generalize by fitting programs that reproduce/predict the target data across more inputs while keeping program size fixed.
More from AGI Musings
- Anil Seth's 'Conscious AI and biological naturalism' collection now open access — anilkseth · 2026-10-10
- AI is laundering human work and knowledge, argues call for idea provenance tracking — asusarla · 2026-10-10
- Spend two years at a Fortune 500 before claiming AI will replace all human labor — burny_tech · 2026-10-10
- Satire about "outraged mathematicians" shows some people actually believe it — burny_tech · 2026-10-10
- AI Is Restructuring Society's Value Hierarchies — Single-Objective Players Become NPCs — GhaffariMaani · 2026-10-10
- Is It Fine to Say AI Has No Consciousness? A Debate Over Whether We're Asking the Wrong Question — asusarla · 2026-10-10