Study: Cross-Version Transfer of Qwen Interpretability Lenses

imstilllearningthis · reddit · 2026-08-16

This study tests whether a Jacobian Lens (interpretability tool) fitted for Qwen2.5-72B can be directly applied to Qwen2.5-72B-Instruct without retraining.

Key Findings:

Reading: The transferred lens performs well in identifying latent entities, with mid-layer rankings even better than the original model. Surface next-token prediction incurs higher costs in deep networks (2x).
Steering: Direction vectors extracted from the old model (e.g., 'paradox') can successfully remove specific concepts in the new model without breaking descriptive coherence.

Conclusion: Cross-checkpoint transfer is viable, allowing monitoring pipelines to reuse old lenses without refitting for every model update.

Original post →

Study: Cross-Version Transfer of Qwen Interpretability Lenses

More from Models