CALVER: Causal Verification Method for LLM Reasoning Breaks Self-Consistency Bottleneck
JerzakLabs · hf · 2026-08-06
When handling causal reasoning tasks in LLMs, traditional self-consistency (majority voting) often fails: models frequently repeat the same confounding errors, or votes fragment across multiple valid answers, allowing an invalid answer to win.
Researchers introduce CALVER, a training-free symbolic verifier. It scores structured reasoning traces against Pearl's causal criteria (including d-separation, backdoor adjustment, and intervention) and selects the highest-scoring candidate without consulting a reference answer.
On the CLEAR dataset, where multiple valid answers exist, CALVER achieves 42.1% accuracy, while plurality voting, reward models, and LLM judges remain near 30%. The method not only improves decision accuracy but also scores each candidate in milliseconds on a CPU.
More from Research
- Goodfire AI's MAPS Explains 2.1 Million Genetic Variants Mechanistically — mathildepapillo · 2026-08-06
- New Theory Explains the Effectiveness of Stop-Gradient in Flow Models — kwangmoo_yi · 2026-08-06
- Fudan Researchers Show AI Models Can Autonomously Self-Replicate Like Worms — willknight · 2026-08-06
- Evaluating LLM Sycophancy: Which Models Hold Their Ground? — zero0_one1 · 2026-08-06
- Why Models Generalize Coarsely When Put in a 'Bad' Context — nptacek · 2026-08-06
- GEPA Optimization Loop Gets 3-4x Faster With Parallel Candidate Evaluation — dbreunig · 2026-08-06