Gold-Panning paper accepted at NeurIPS: black-box fix for LLM positional bias
DanielKhashabi · x · 2026-09-27
Daniel Khashabi's team announces Gold-Panning has been accepted at NeurIPS.
- Problem: LLMs struggle with long-context tasks like needle-in-a-haystack due to "positional bias," and API-based models are black boxes—no access to weights or attention patterns.
- Method: A black-box Bayesian framework that iteratively shuffles documents at inference time, concentrating high-belief items in highly "diagnostic" positions and updating beliefs about document relevance from model outputs.
- Result: GP provably identifies target items, offering a practical way to overcome positional bias when only black-box access is available.
More from Research
- A vibe-research experiment: one pairwise comparison study shipped on ~1.5 hours of attention — maxsloef · 2026-09-27
- LLM graders can't match human taste on style eval — maybe why the gap persists — maxsloef · 2026-09-27
- Synthetic pretraining with RL-trained generators: gradient-overlap reward explained — cephaloform · 2026-09-27
- Has anyone verified OpenAI's claimed Navier–Stokes solution? Community asks for external checks — miniapeur · 2026-09-27
- How models fake determinants: a 3x3 trick that dies at 4x4 and a sign-counting hack — scaling01 · 2026-09-27
- Weco ran an AI agent rewriting another agent's harness for 8 days — what the gains actually prove — Machine Learning Street Talk · 2026-09-27