Correlated verifier cascades can make LLM reliability plateau below 100%
Jiangang Han · hf · 2026-07-21
Partially correlated verifier cascades can hit reliability ceilings
This note studies serial verification gates in LLM harnesses, where an answer is returned only if all k verifier calls accept it.
Main results:
- Under latent correlation between verifier outcomes, the exact posterior becomes ellk = ell0 - ln mk, where mk is the k-th moment of a latent false-accept rate.
- The log-odds curve is concave in k, so the independence-based Odds Law is only a tangent/upper bound, not the general case.
- For Beta latent variables, failure decays polynomially rather than exponentially.
- If some errors are effectively blind spots, reliability can saturate below 1 no matter how many gates are added.
- When true-accept rates also vary, the system can eventually improve, plateau, or even get worse depending on tail behavior.
The practical takeaway is that decorrelation — changing model family, modality, or evidence source — matters more than simply adding more gates.
More from Research
- Could 10k agents discover learning methods beyond backprop, or just tweak existing ones? — SeunghyunSEO7 · 2026-09-11
- Cognition's SWE-2 uses a KKT duality argument in RL to shift the effort Pareto curve — YouJiacheng · 2026-09-11
- VidMap uses RoMa coarse matching on all frames, fine-scale only for keyframes — ducha_aiki · 2026-09-11
- Bug Hunt Bench author: leaderboard noise is about 2-3 points — PawelHuryn · 2026-09-11
- PNAS paper shows a tiny billiard-ball system is a universal computer — undecidability lives in two dimensions — eigensteve · 2026-09-11
- New paper: Absolute pose estimation from affine cues and gravity direction — ducha_aiki · 2026-09-11