Auditing Predictive Models via Internal Representations

eternisai · hf · 2026-07-16

Research Highlights

This work discusses how the "thinking process" of predictive LLMs may not faithfully reflect their true underlying basis, whereas internal activation representations might reveal a model's judgment state more directly than Chain-of-Thought (CoT).

Key Findings

The authors trained representation-pooling probes based on intermediate activations on Eternis-Forecaster 8B and OpenForesight, finding their calibration significantly outperforms conventional CoT explanations.
This conclusion also holds true on GLM-4.7-Flash and GLM-4.5-Air.
Through evidence ablation and interference injection tests, the authors discovered that removing key sources from the prompt alters the model's predictions, but the reasoning chain often remains unchanged, suggesting CoT may be unfaithful.
Acting like a "lie detector," these probes can better track behavioral changes; they can also predict the direction of change in 84% of cases, even when the perturbation's impact is invisible in the CoT.

Reasoning and Efficiency

The paper also notes that forcing a single "answer" before formal reasoning generation often recovers the model's final answer and confidence level.
Routing questions based on the dispersion of this "preset answer distribution" can save 30%–47% of generated tokens without sacrificing accuracy.

Conclusion

The authors argue that probing internal representations is a practical tool for calibrating, auditing, and routing language model prediction tasks, offering greater reliability than simply relying on CoT.

Original post →

Auditing Predictive Models via Internal Representations

More from Research