The Linear Representation Hypothesis Needs a Group Action
Louie Hong Yao, Yuhao Li, Shengchao Liu
cs.LG, cs.AI, cs.CL
2026-09-23
Yao, Li and Liu specify Linear Representation as (G, M, F, P). Probe weights live in V* and steering vectors in V; one refusal vector used two ways tests two hypotheses.
Interpretability work talks about "linear representations" as if that named one object. Linear probes, difference-in-means steering, sparse autoencoders, and representation similarity all borrow the phrase. Yao (independent) with Li and Liu at CUHK argue the missing piece is equivalence: when two realizations count as the same representation.
Arditi et al. give a clean illustration. They estimate one difference-in-means vector and spend it twice. Added without normalization, it is a displacement in V, and its length changes the intervention. Normalized and projected out, only its line in projective space P(V) matters. Both interventions work, and both are cited as evidence for a single refusal direction. The array is the same. The mathematical object is not.
Neither use is invalid. The trouble is that estimation, comparison, and intervention often switch objects without saying so, while the write-up treats them as one claim.
A representation claim is a 4-tuple (G, M, F, P). G is the equivalence group: which coordinate changes are mere redescription. M is the space the extracted object lives in, together with how G acts on it. F is the procedure that builds the object from a representation. P is the predicate asserted of that object. F must be equivariant and P invariant, otherwise the conclusion is about a coordinate chart.
The groups used in practice nest: Giso ⊂ Gsim ⊂ Gaff. Affine maps keep collinearity, subspace dimension, and incidence, not angles or lengths. Similarities (positive scale times an orthogonal map) keep cosines. Isometries freeze scale, so norms become meaningful. A larger G is a stronger claim about the representation and a weaker claim about any particular number.
The object can change even when everyone says "linear." Displacements (word-vector analogies, difference-in-means, Contrastive Activation Addition) live in V and transform as v ↦ Av. Probe weights live in the dual V and transform as w ↦ A^{-⊤} w. Superposition and SAEs decompose an activation as a sum of feature directions: the activation-side story is affine-covariant, while the latents have a separate mixing ambiguity. A 1-D feature lives in P(V); a k-D feature lives in the Grassmannian Gr(k, V). Circular day/month features are the 2-D case.
Proposition 3.2 is the identification ban: for dim V ≥ 2 there is no nonzero GL(V)-equivariant map V → V. An inner product gives such a map only under isometries; passing to projective directions enlarges the symmetry to similarities. An ℓ2 penalty on a probe has the same effect, cutting equivariance down to Giso. Difference-in-means and a leading PCA direction can both land in P(V) while obeying different groups, because PCA needs an inner product to rank variance and difference-in-means does not.
G is bounded from below by the architecture. Function-preserving reparameterizations already act on internal activations. In dot-product attention, WQ ↦ Λ WQ and WK ↦ Λ^{-⊤} WK leave the model function unchanged for any invertible Λ on the head. At the residual stream that gauge acts trivially. At a query or key site it fills GL(dhead), which admits no nonzero invariant bilinear form, so angles and norms from a fixed inner product are not properties of the model. RoPE shrinks the gauge to independently scaled rotations of the rotary planes. The real dimension is still dhead. Independent per-plane scales still move norms, cosines, PCA directions, and log det(KK⊤).
Pipelines inherit the strictest stage unless the composite is checked as a whole. A direction estimated under Gaff, compared by cosine under Gsim, and scaled by a norm under Giso is guaranteed only under Giso. Centering is the exception that proves the point: cosine after subtracting the mean can ignore translations that would change cosine on raw activations.
There is no new benchmark score. Table 1 sorts common quantities by the largest group that preserves them: 4 under Gaff (exact rank, span, CCA, subspace containment), 9 under Gsim (cosine, leading principal subspace, relative-threshold rank, effective rank, intrinsic dimension, principal angles, Grassmann distances, linear CKA, nearest-neighbour agreement), 5 under Giso (norm, intervention magnitude, absolute-threshold rank, Procrustes, reconstruction loss). Dictionary learning sits on two spaces at once. Euclidean reconstruction on the activation side forces Giso; a coordinatewise sparsity penalty on the latents cuts mixing symmetry to signed permutations.
Case audits fill in the practice. Bhalla et al. treat probe weights and steering vectors as additive intervention directions and compare them by cosine. Cosine between projective directions can be meaningful under Gsim once a metric is chosen. Reusing the probe coefficients as a displacement is equivariant only under Giso. CAA normalizes magnitudes across behaviors but not across layers; Activation Addition leaves the vector raw. The same numeric coefficient is not the same displacement. Wollschläger et al. rescale an optimized refusal direction to the difference-in-means norm, which at least names the Euclidean scale.
KeyDiff evicts KV-cache entries by pairwise cosine among keys in a head, and the supporting plots report key/query cosines, L2 norms, PCA, and log det(KK⊤). The eviction rule can still work. Those geometric explanations are not invariants of the computation they purport to explain. Gauge-related parameters compute the same function and can select different tokens under cosine. The appendix writes an invariant substitute: with query second moment Ω, d(k1,k2)² = (k1-k2)⊤ Ω (k1-k2) is the mean squared change in attention logit from swapping keys. When Ω ∝ I this collapses to Euclidean distance. Yamagiwa et al. compare column spaces of transposed projection matrices in the residual stream; WQ ↦ Λ WQ leaves that span alone, which matches the architectural symmetry.
Inference-Time Intervention uses probes to pick heads, then compares probe-weight, mass-mean-shift, and CCS directions as intervention candidates. The added displacement is a normalized direction times the empirical standard deviation along it. Under A = sQ that calibrated vector transforms as A(σŵ), so the full construction is compatible with Gsim; identifying a dual probe with a primal displacement still depends on the chosen similarity geometry. SVCCA prepends singular-value truncation to CCA. CCA on centered representations is affine-invariant; the spectral cutoff is not, and later CCA cannot put that invariance back.
The same gap shows up in the Platonic Representation Hypothesis. Models of different width do not share a vector space, so comparison runs through neighborhoods or similarity matrices on a shared input distribution, and the claim is indexed by that distribution. Kornblith et al. already showed that if width is at least the number of sampled inputs and both activation matrices have full row rank, a similarity invariant to all invertible linear maps cannot tell the two representations apart. An isometric reading would demand absolute metric agreement that different widths, tokenizers, and objectives generally cannot meet. Current evaluations sit near Gsim, with the effective group chosen by the alignment metric rather than stated on its own.
This paper does not ship a method. It ships a reporting rule: name the G-space, why F is equivariant, what P asserts, and where the representation is read. Group-theory notation is optional. Recoverability from the claim is not.
The cost is real. Some conclusions shrink once the geometry is written down. Some comparisons need an extra equivariant map. Some numbers belong to a representation plus a chosen metric or parameterization, not to the representation alone. Declaring Giso everywhere is tempting and forbidden when the architecture already realizes maps outside Giso.
The practical split is sharp. Cosine in the residual stream and cosine at a Q/K site are different kinds of statement. The second is about a parameterization, not about the model function. Copying a direction across layers with the identity because both residual streams have dimension d fails for the same reason.
Admissibility (A1) is cheap: shrink G or weaken P. Content lives in maximality, the largest group under which the claim still holds. That is why Table 1 is sorted by largest preserving group.
This is a position paper. It does not measure how many published conclusions would recede once equivalence is named. The case list is selective: refusal directions, CAA, Activation Addition, KeyDiff, ITI, SVCCA. It is not a census of interpretability.
The main hierarchy is three affine-type groups. Training also produces permutation symmetries, scale-and-shift normalizations, and narrower orbits along the optimization path; those stay off the main table. The RoPE appendix computes the key-site gauge exactly and does not report, on a large model, how often cosine eviction and the invariant d pick different tokens. It only shows that they can.
A second gap is empirical. Treating the geometry of the trained θ as a fact about the artifact in hand is allowed. That fact is about θ and the training procedure, not about fθ. Attributing it more broadly needs stability across seeds and runs. The paper flags that this check is rare and does not run it.
There is also no algorithm that estimates the maximal admissible group. Applying the audit is still a manual lookup against Table 1.