Paper: LLMs severely underestimate missing information, study finds
marinkazitnik · x · 2026-08-19
This study formalizes multi-turn information seeking as a k-underspecified constraint satisfaction problem, creating the MT-INFOSEEK benchmark across math, logic, and biology.
Key findings:
- Detection vs. Quantification: Models detect missing info but fail to gauge how much is missing. Performance drops as underspecification increases; in logic problems, they underestimate missing info 4x more often than overestimate.
- Identifying Needs: Models struggle to select a minimal sufficient set of variables. Knowing the true k helps only modestly, with many interactions ending before the target is determinable.
More from Research
- Anthropic Open Sources Prompts: Claude Achieves 35% Hit Rate in Protein Design — Hesamation · 2026-08-19
- Choose Your Research Hill: Don't Be the 'Hinting-based Self-Distillation' Person — nrehiew_ · 2026-08-19
- Transformer Byte-Level Embeddings Clash with Kernel Factorization — kalomaze · 2026-08-19
- Datoric launches research division focusing on speech and video understanding — ycombinator · 2026-08-19
- MCP ecosystem audit: 47 of 163 popular Reddit servers unregistered — mcpindex · 2026-08-19
- Claude achieves over 30% success rate in de novo protein design — AnthropicAI · 2026-08-19