TTCA metric and LAAR routing: measuring time until the first correct answer, not just latency
burkov · x · 2026-11-03
- Conventional latency metrics miss a hidden cost: a fast but wrong answer still costs users time through retries, so accumulated delay is underestimated.
- The authors propose Time-to-Correct-Answer (TTCA): timing from the first attempt until the first correct answer, and ask whether routing by predicted accuracy plus expected latency can reduce it.
- Experiments: five models on automatically scored key-value retrieval tasks with contexts from 4,000 to 64,000 tokens in English, Japanese, and Chinese.
- They build Lightweight Accuracy-Aware Routing (LAAR), using simple features like language and prompt length to estimate each model's success probability and latency, then route requests to minimize TTCA.
More from Research
- SJTU-led paper proposes 'Avatar Intelligence' and an Agentic Oversight Index for personal agents — SJTU · 2026-11-03
- Vapor-cell atomic E-field sensor hits 0.2 mV/m/√Hz in 11 mm³, no electrodes — jwt0625 · 2026-11-03
- Anthropic Open-Sources FlashProve, Zero-Knowledge Proofs of AI Inference, Tested on Llama 4 Maverick — yenkel · 2026-11-03
- BiGym 2.0: Claude-written policies hit 51.9% on humanoid household tasks, nearing VLA — teortaxesTex · 2026-11-03
- Hopfield model thermodynamic limit proven at all loads without replica-symmetry assumptions — _onionesque · 2026-11-03
- 3D World Simulator debuts: explicit 3D representation for long-horizon rollouts — Scobleizer · 2026-11-03