Correlated verifier cascades can make LLM reliability plateau below 100%
Jiangang Han · hf · 2026-07-21
# Partially correlated verifier cascades can hit reliability ceilings This note studies serial verification gates in LLM harnesses, where an answer is returned only if all `k` verifier calls accept it. Main results: - Under latent correlation between verifier outcomes, the exact posterior becomes `ell_k = ell_0 - ln m_k`, where `m_k` is the `k`-th moment of a latent false-accept rate. - The log-odds curve is concave in `k`, so the independence-based Odds Law is only a tangent/upper bound, not the general case. - For Beta latent variables, failure decays polynomially rather than exponentially. - If some errors are effectively blind spots, reliability can saturate below 1 no matter how many gates are added. - When true-accept rates also vary, the system can eventually improve, plateau, or even get worse depending on tail behavior. The practical takeaway is that decorrelation — changing model family, modality, or evidence source — matters more than simply adding more gates.
More from Research
- Baseten study finds new facts in LLM weights are fragile unless trained from many restatements — alex_verem · 2026-07-21
- Kimi K3 and Fable 5 now look much closer than the old open-vs-closed gap — FinanceYF5 · 2026-07-21
- uv-scripts/ocr returns to the top of Hugging Face datasets with a JSON model picker — vanstriendaniel · 2026-07-21
- DeepSearch-World trains web agents with 420K verifiable QA tasks — HKUST · 2026-07-21
- GigaAM Multilingual targets low-resource Central Asian ASR with 2M hours of audio — ai-sage · 2026-07-21
- WorldCupArena benchmarks language models on 104 football matches — Zhaokai Wang · 2026-07-21