Beyond ICA: Identifiability by Symmetry Breaking
Pengzhou Wu
stat.ML, cs.LG, math.ST
2026-07-25
Unsupervised identifiability for PWA-decoder, GMM-prior DGMs: three algebraic contrasts recover latent law, decoder, and posterior without continuity, injectivity, or domain labels.
Without identifiability, two training runs on the same data can converge to different mechanisms, and downstream tasks lose well-posedness. Nonlinear ICA breaks that ambiguity with observed domain labels: TCL uses time segments, iVAE uses auxiliaries. Real data often has neither. Modeling unobserved domains as GMM components is qualitatively harder, because domain contrast alone no longer rules out spurious solutions.
Deep generative modeling needs the decoder and the posterior, not pointwise source recovery. Classical ICA inherited injectivity from signal processing. TCL and iVAE style training almost never enforce injectivity, even when the theorems assume it. Kivva et al. opened unsupervised PWA+GMM identifiability, yet map identifiability still wants global injectivity, and density matching for that step leans on continuity. Pengzhou Wu at SANKEN, University of Osaka, studies X = f(Z) with Z an irredundant GMM and f a reduced piecewise-affine map. Only the law of X is seen. The question is whether that law recovers f and the law of Z up to a named ambiguity class.
The engine is algebraic symmetry collapse. When two candidate models share an observation law, the mismatch is a nontrivial element of the latent law's affine symmetry group. Trivializing the group forces the models to coincide. Log-density and Jacobian comparisons are replaced by group triviality, so discontinuous maps and chamber boundaries become usable. A continuous PWA map already carries the branch-boundary structure. Discontinuity is permitted; the driving resource is the combinatorics of those boundaries.
Three contrasts:
The observation space is cut into chambers, maximal open connected sets with a constant active-branch index. On each chamber the pushforward is a finite GMM. Agreement on an open set, analytic continuation, and linear independence of Gaussian densities give a global bijection on branch-component pairs. One simple boundary yields an affine link α (law identifiability). USB gives every branch its own simple boundary; MST sets the branch-difference maps to the identity, yielding map identifiability: f = g ∘ α^{-1}. Theorem 4 shows why PI is required: a strict PWA self-map S of the GMM would produce g = f ∘ S, a different mechanism with the same observations. Under PI, any such self-map must be globally affine. Injectivity is reserved for optional pointwise inversion. The main theorems allow full component covariances.
There are no training runs and no numerical comparison with iVAE or Kivva. The hierarchy is theoretical.
| Level | Truth-side conditions | Conclusion |
| LID (Thm 5) | (PI)+(SB) | α(Z') ∼ Z for a global affine α |
| MID (Thm 8) | plus (USB)+(MST) | f = g ∘ α^{-1} |
| Posterior (Prop 9) | follows from MID | posteriors correspond via α; injectivity not required |
| Pointwise (Prop 10) | plus global injectivity of f | z = α(ẑ) |
Under (DD), Corollary 11 sharpens α to the ICA form PΛz+c: component covariances are diagonal, and two components have pairwise distinct per-coordinate spectral ratios. Diagonal covariances are not coordinate independence. A 2D example in the paper uses identity covariances inside each component and still has Cov(Z1,Z2)=9/4.
TCL and iVAE need observed labels and smooth invertible generators. Kivva needs global injectivity for MID and continuity for density matching. Xu et al. stay with injective continuous PWA. This setting allows discontinuous f and fully non-injective f, where every observation has multiple latent codes. All conditions are generic under unconstrained parametrizations.
Identifiable representation learning need not use a normalizing flow and need not wait for domain labels. ReLU networks are already continuous PWA maps; the boundary structure is the resource. Posterior identifiability still holds when fibers contain several latent codes, which is closer to how VAEs are used.
The training prescription is thin. USB, MST, and SB live on the data-generating process. The estimator needs only (PI). Architecture does not have to encode the rest. Causal uses, matching Intact-VAE's shared affine ambiguity across treatment arms, and multi-mechanism discovery in the LiNGAM spirit, are prospects, not algorithms.
This relaxes identifiability assumptions. It is not a new generative model.
The three contrasts are argued to be structurally necessary. Without (SB), a fold construction in Appendix K.1 breaks LID even when (PI) and (MST) hold. Without (MST), any nontrivial affine self-symmetry contains a finite-order torsion that can be localized into a strict PWA self-map. The gap between global and chamber-wise (PI) remains a conjecture.
Generic means full measure in parameter space, not that a trained ReLU generator satisfies USB. There is no experiment that checks unique boundary toggles on real networks, and no regularizer or diagnostic for (PI). The main theorems are noiseless pushforwards X=f(Z); additive noise is a footnote pointing at deconvolution. Equal-weight mixtures, common in practice, break the distinct-weights sufficient condition for MST; the main theorem uses MST itself, and no empirical frequency is given. Extending the engine from GMM+PWA to other exponential families is left open.