IBM's STAIR treats a book's table of contents as the index: 82.6% recall, 65x lower hallucination
alex_verem · x · 2026-09-26
IBM Research's STAIR paper rethinks retrieval for long documents:
- Problem: standard RAG chunks documents into equal-sized pieces for vector search, throwing away chapters, sections and headings — your 500-page manual becomes a pile of loose paragraphs
- Method: STAIR keeps the structure. The model reads the table of contents and learns to answer each question with the right section title — the outline becomes the index, skipping embeddings and vector databases
- Benchmark: SearchTome, built from 18 books across law, medicine, finance, education and the sciences, with tens of thousands of questions
- Results: 82.6% Recall@1 vs 76.9% for the best comparable method, 68.7% dense retrieval, 59.5% keyword search, 13.8% off-the-shelf Mistral; hallucination rate of 0.05% vs 3.25% for the closest method — the source of the "65x" claim
The author also flags limits viral posts skip: STAIR needs a document with a ToC, fine-tunes a separate 7B model per book (200 passes each), is untested on company-sized collections, and the paper doesn't check whether answers built from the section are correct.
Core insight: the chapter structure authors spent months organizing is the best index you already have — and most AI search ignores it.
Related event: IBM's STAIR Uses Book TOCs as Index, Slashing Hallucinations 65x vs RAG(3 posts)→
More from Research
- Professor builds 'golf' game with word embeddings to teach cosine similarity — prof_g · 2026-09-26
- Benchmark maker says no Ascend version — models would hill-climb it; TPU/Trainium/AMD better — xeophon · 2026-09-26
- RBC Borealis Details ATOM, Its Large Transaction Model Wired to LLMs via Agentic Workflows — VectorInst · 2026-09-26
- Kelsey Allen's 3DSPA Brings Human-Like Physical Realism Scoring to Generative Video Models — VectorInst · 2026-09-26
- At OpenAI's 2026 AGI Economy Conference, researchers call for economic-task evals — soumitrashukla9 · 2026-09-26
- AI could codify tacit organizational knowledge, worth up to one US GDP, paper says — soumitrashukla9 · 2026-09-26