Amazon's SpanUQ brings span-level uncertainty estimation to LLMs in a single forward pass
机器之心 · wechat · 2026-10-22
Amazon researchers present SpanUQ at NeurIPS 2026, introducing a new task called Span-Level Uncertainty Estimation (SLUE): identifying semantically complete spans in LLM output and estimating uncertainty for each, so downstream systems can make per-claim decisions instead of judging whole responses.
Motivation
- Token-level uncertainty is fine-grained but not semantically meaningful; sequence-level scores can't localize which part of an answer is unreliable.
- In multi-agent systems, passing per-span uncertainty between agents lets receivers decide when to verify or escalate.
Method
- A 25M-parameter probe reads frozen LLM hidden states (multi-layer fusion) and predicts spans plus uncertainty in one forward pass, avoiding costly multi-sample estimation.
- A DETR-style span decoder with learnable queries detects overlapping spans in parallel, reaching 0.910 F1, a 39.4% relative gain over the strongest heuristic.
- Mixture of Beta distributions models continuous uncertainty, trained with Beta NLL plus contrastive ranking objectives; uncertainty-conditioned iterative refinement adds <15% overhead while improving ranking correlation.
Benchmark
- SpanUQ-Bench: 20K prompts, 293K annotated spans with continuous soft labels built from multi-sample claim verification.
- Validated across five LLM backbones spanning dense and MoE families.
More from Research
- EMNLP paper: diffusion LMs internally encode latent denoising timestep representations — gsarti_ · 2026-10-23
- KU Leuven and VIB open two Group Leader professorships in AI for Biology, start 2027 — jpillowtime · 2026-10-23
- Subspace Tuning curbs LLM catastrophic forgetting by freezing principal singular directions — scottniekum · 2026-10-23
- Perceptic launches PercepticOS, an AI workspace for 'scientific superintelligence' — nathanbenaich · 2026-10-23
- EPFL's Bosselut to talk long context as test-time learning at ELLIS seminar — andre_t_martins · 2026-10-23
- DRATS: Adaptive Task Sampling Boosts Worst-Task Performance in Multi-Task RL — Cohere · 2026-10-23