NYU's CIA Metric Finds LLMs' Chain-of-Thought Only 44.8-75.9% Aligned With Internal Computation
newyorkuniversity · hf · 2026-10-07
NYU researchers measure and improve the alignment between LLMs' chain-of-thought and their internal computations.
Key contributions
- CIA (CoT-Interpretability Alignment) metric: measures agreement between CoT traces and internal reasoning strategies detected by interpretability tools
- Evaluated on two-hop QA, hint intervention, and integer multiplication across three LLMs, finding limited alignment (44.8-75.9%)
- Post-training with both task accuracy and parametric faithfulness as rewards substantially improves CoT faithfulness while maintaining or improving task accuracy
- Code and data open-sourced: github.com/yihuaihong/CIA-minimal-repro
The work provides both an auditing framework for CoT faithfulness and a pathway toward more trustworthy explicit reasoning.
More from Research
- Harvard paper: higher SWE-Bench scores didn't translate into more merged pull requests — rohanpaul_ai · 2026-11-23
- Princeton's SSoN Benchmark: Best Vision Model Scores 39.7% vs Human 83.3% on Spatial Consistency — jiqizhixin · 2026-11-23
- Four Color Theorem Was Exhaustion — This Lean Proof Is Different, Argues Mathematician — gleech · 2026-11-23
- Samsung Labs open-sources LittleBit, compressing LLM weights to 0.1 bits — MikePFrank · 2026-11-23
- Mathematicians Debate: Can 60GB of Lean Code Turn Into Real Understanding? — gleech · 2026-11-23
- Adversarial examples aren't ubiquitous: budget matters, argues researcher — maksym_andr · 2026-11-23