Bengio et al. new paper: quantifying compositional structure via algorithmic information theory
patrickmineault · x · 2026-10-27
A new paper by Yoshua Bengio, Blaise Agüera y Arcas, Eric Elmoznino et al. gives an intrinsic definition of compositionality using algorithmic information theory: compositional structure is the 'novel shared structure' that appears when modeling the whole beyond what the parts explain (random strings score zero). Built on Kolmogorov complexity and minimal sufficient statistics, it conjectures the true part-split is the one maximizing the score. CPPN generation experiments show strings evolving deeper grammars and images developing repeated motifs under a combined Structure + Compositionality reward, turning compositionality from an assumption into a measurable quantity.
More from Research
- Looped transformer analysis pins down single neurons for non-duality, phonetics, and other concepts — burny_tech · 2026-10-27
- LLMs negotiate well in deterministic settings but only 3 of 9 pairings close deals under high stochasticity — davidad · 2026-10-27
- Single-neuron probes of Qwen3.5 surface a cast of hidden personas, from Naruto to cartoon mascots — burny_tech · 2026-10-27
- Sakana AI RSI Lab showcases Schmidhuber keynote, Japan-built foundation model Sawara, and open-model momentum — Stefania_druga · 2026-10-27
- ETH proposers lead open review of solver claim; exponent shrink factor ~0.67 at a billion variables — AIFlow_ML · 2026-10-27
- PSI's AI research system cracks five open quantum information problems, with Lean proofs — IgorCarron · 2026-10-27