LexFrame: interpreting LM residual streams in the model's own semantics
Jian Gu · hf · 2026-10-19
A new paper, Lexical Reference Frames, proposes LexFrame for interpreting LM residual streams: instead of fixed-vocabulary projections like the logit lens, hidden states are interpreted through the model's own remaining layers, using the model's native semantics as the reference frame.
The method enables accurate attribution, hidden-state steering, and early layer exit for cheaper inference. The paper's diagrams show LexFrame's per-layer angular deviation from the final output direction is far smaller than logit lens or Jacobian lens.
More from Research
- Niels Rogge launches paper feed: follow AI research by task, method, author or lab — NielsRogge · 2026-10-19
- A frozen 16MB vocabulary replaces learned embeddings in small and ternary LMs — Sai Kiran Bathula · 2026-10-19
- Meituan LongCat releases RAP: 25K paper-poster pairs as a visual coding benchmark — meituan · 2026-10-19
- BELIEFWEAVER: belief updates and world modeling for long-horizon agent failure recovery — IntelligenceLab · 2026-10-19
- Consistent Distribution Matching: data-free diffusion distillation beats DMD2 with 20GB VRAM — vector-institute · 2026-10-19
- DeepMind's XYEval: one bad user hint drops agent success from 67.4% to 36% — i_dg23 · 2026-10-19