Gemma study finds physical laws are encoded as relationships between hidden states
ProfBuehlerMIT · x · 2026-07-23
Using Google's Gemma model, the authors argue that physical knowledge is stored in relationships between hidden states, not in static activations.
They probe the model with counterfactual prompt pairs and identify three layers of internal physics:
- Readability: broad materials concepts such as corrosion, toughness, and oxidation are linearly recoverable from intermediate hidden states even when the words are absent from the prompt.
- Representation: state displacements, rather than absolute positions, track direct/neutral/inverse constitutive laws across 60 materials-science laws with ρ = 0.910, and correctly orient 39 of 40 directional laws.
- Causal use: injecting a frozen microstructural direction into hidden states shifts output preferences in a controlled, law-consistent way.
The paper frames this as a step toward representation-aware scientific AI: models that preserve physical laws internally and expose testable reasoning structures instead of relying on shallow text shortcuts.
Related event: Physics Knowledge in LLMs Lies in Hidden State Transformations(2 posts)→
More from Research
- Hugging Face releases The Stack v3, a 5T-token open code dataset — lvwerra · 2026-07-23
- Mila Quebec will host a talk on what AI benchmarks really measure for African languages — hugo_larochelle · 2026-07-23
- An agent stack diagram says production AI is 90% architecture, not prompts — theomitsa · 2026-07-23
- MIT’s free “SLAM for Dummies” guide turns robotics navigation into a hands-on tutorial — lukas_m_ziegler · 2026-07-23
- Sol 5.6 reportedly writes a full research paper from one prompt — conitzer · 2026-07-23
- NeurIPS position paper reviews are now out — hiddenmarkov · 2026-07-23