Step-by-step prompting makes LLMs worse at skill tagging, study finds
Researchers from three US universities found that chain-of-thought prompting dropped LLMs' Q-matrix tagging accuracy on fraction-subtraction problems from 87.7% to 80.9%. The work is published on arXiv as part of a study on reasoning-trace analysis for interpretable Q-matrix construction.
2026-11-21 ~ 2026-11-21 · 2 related posts
- LLM chain-of-thought hurts Q-matrix accuracy: 80.9% vs 87.7% expert match, study finds — alex_verem · 2026-11-21
- Reasoning-Trace Analysis paper: auditing LLM-generated Q-matrices via CoT traces — alex_verem · 2026-11-21