The information geometry of large language models is shared, learned, and controllable
Dario Picozzi
cs.LG, cs.CL
2026-09-10
Next-token Fisher geometry is fixed by behaviour. Rank agreement is 0.88 across ten models vs 0.62 in activation space; GPT-2 edit off-target KL drops about tenfold.
Most steering, feature-importance and fine-tuning regularisers treat Euclidean distance in activation space as if it were part of the model. It is not. The same input-output map can be realised in any invertible linear coordinate system, and Euclidean lengths move with that choice. Next-token distributions do have a canonical metric. Chentsov's theorem says that, up to a global scale, Fisher-Rao is the unique Riemannian metric invariant under maps that preserve statistical information. Behaviour space has a privileged geometry; representation space does not.
This UCL paper pulls the output Fisher back to an intervention layer and uses it for three questions: whether independently trained models share that geometry, whether language statistics predict its shape and the timing of fact acquisition, and whether updates that respect it disturb other predictions less than Euclidean steps.
Map an intervention-layer activation h to the next-token law p. The output Fisher is the covariance of unembedding rows under p; pulling it back through the network Jacobian gives G. For a small step, KL is 1/2 δhᵀGδh plus a cubic remainder. Given an objective gradient q, the damped natural-gradient step (G+αR)⁻¹q is the minimum regularised output-cost move that achieves a stated first-order change.
Cross-model comparison needs no shared tokenizer or hidden basis. On a common battery of contexts, each model yields a matrix of Fisher-Rao distances among its next-token laws; Spearman correlation of the upper triangles asks whether they rank the same pairs as near or far. A controlled language assigns identical facts to shallow or deep evidence constructions and measures acquisition time. Intervention tests match the target change and compare off-target KL of Euclidean versus natural-gradient steps.
Ten models span transformer, state-space and recurrent stacks, 70M to 7B, seven training pipelines and four tokenizers. Mean rank agreement of output geometry on natural text is 0.88, against 0.62 for mid-layer activations and 0.61 for last-layer activations. Mapping each next-token law onto the first remaining byte keeps cross-tokenizer agreement at 0.91. The shared component carries semantics: an eight-way probe transfers at 0.66 accuracy, 0.70 through consensus geometry alone, 0.72 within model, 0.125 chance. On synthetic languages, changing the assigned conditional law recovers more than 99% of the imposed squared geometric gap; swapping architecture inside a law barely moves the geometry.
Human completions follow predictive fit. Semantic squared distance on Pythia falls from about 0.34 at 70M to 0.30 at 2.8B. A risk-to-alignment curve fit on Pythia predicts OLMo checkpoints and five external models on a 384-context battery without refitting (correlation 0.76, RMSE 0.029).
Output-Fisher eigenvalues decay roughly as i⁻¹ over ranks 22-80. A token-probability profile with no fitted parameters predicts effective dimension, with family-median RMSE 0.027-0.052 on five held-out families. Corpus n-gram margins measured before training predict held-out fact trajectories (R² 0.775-0.792, timing error 0.77-0.96 log₂ steps). Randomly assigning the same facts to deeper evidence delays acquisition-time quantiles about 4.3 times; across GPT-2, GPT-NeoX and Llama-style decoders the delay is 1.75-2.86 times.
On 3,515 matched measurements across 11 models, six families and three objectives, the finite-damping cost ratio has median measured/predicted 0.967. Along relinearised paths, Fisher cumulative local KL is 11.5-127 times smaller than Euclidean. Updates learned on four donor prompts transfer to unseen prompts and cut reference-sequence KL by 2.9-6.3 times versus Euclidean control. The same correction covers editing, attribution, dictionary learning and fine-tuning:
| Operation | Main number vs Euclidean / Adam |
| Factual steering | up to 72.7 times less off-target KL |
| GPT-2 CounterFact edits | 10.57 times less other-token KL (geometric mean) |
| SAE feature ablations | Fisher cost correlates 0.997 with exact KL |
| Natural-gradient LoRA | 1.57 times better preservation-KL frontier than AdamW+KL at 70M |
Alignment is two questions: which behaviour to change, and how to change it without dragging everything else. This paper answers the second. Given an externally supplied objective direction, output geometry gives the locally cheapest activation step, and the cost ratio can be computed before the intervention, with no fitted scale. For steering, knowledge editing and preservation-constrained fine-tuning, the move is to replace the Euclidean inner product with this pullback metric. It is not a new objective.
Most models sit between 70M and 7B; the headline editing result is 30 CounterFact records on GPT-2. The local quadratic approximation fades with step size: model-balanced measured/predicted geometric means drop from 0.918 to 0.755 across three intervention magnitudes. The LoRA comparison shows local preservation, not safety retention. Forming G needs Jacobian-vector products; conjugate gradient is at most 85 iterations at the stated relative damping, independent of width, but still not free. Geometry does not choose the target behaviour. Human comparisons use completion norms, not full use.