Deep Sets Revisited: The Theorem That Let Neural Nets Ignore Input Order
burny_tech · x · 2026-09-03
A look back at Zaheer et al.'s "Deep Sets": neural nets once only understood ordered vectors, but real data (point clouds, particle systems, galaxy clusters) is unordered. The paper proved any permutation-invariant function must decompose as ρ(Σφ(x)), deriving the architecture directly from the theorem; for equivariance, the weight matrix must be λI + γ11ᵀ—the only linear op commuting with permutation.
More from Research
- LeVJEPA: video encoder matches V-JEPA 2 with 5.6-20.8x less pretraining compute — CSProfKGD · 2026-09-03
- Astra's rumored looped transformer gets a technical debunk, with Oriol Vinyals citing Universal Transformer — OriolVinyalsML · 2026-09-03
- Hand-crafted Hessian diagonal estimator derivations: novel LLM training data? — Aiden_Tech_Ai · 2026-09-03
- Nature Perspective: explainable AI plus causal reasoning enables learning from models — burny_tech · 2026-09-03
- NeurIPS 2026 medical agents workshop recruits volunteer reviewers — mdredze · 2026-09-03
- New paper: a Lagrangian view explains why straight flows enable fast sampling — burny_tech · 2026-09-03