Hierarchical Indexing and Context Reranking in RAG
bigaiguy · x · 2026-07-11
This thread shares two RAG engineering techniques:
- Context-Aware Reranking
- Retrieve candidates first, then use a lightweight reranker to score them based on the full conversation history rather than just the current query.
- Chunks that contradict prior context are down-weighted, while those that align with it are up-weighted.
- The author claims this reduced contradictions in the final output by 28%.
- Hierarchical Indexing
- Instead of a single index, it uses three:
- Summary index: High-level document summaries.
- Chunk index: Standard paragraph-level retrieval.
- Sentence index: Finer-grained factual retrieval.
- The query router targets the appropriate layer based on the question type: broad queries hit the summaries, specific facts hit the sentences.
Overall, the emphasis is that RAG quality depends not just on "how much is recalled," but also on index hierarchy design and whether reranking understands the context.
More from Research
- Lab monkeys at $30-80K each: a 4-year breeding cycle is the real preclinical drug bottleneck — zebird0 · 2026-10-27
- OpenAI Foundation funds UNC tumor antigen project, hiring senior experimental scientist — iskander · 2026-10-27
- Steering attention heads toward honesty: up moral patienthood, down psychopathy — edelwax · 2026-10-27
- Are transcoders overkill? New paper argues plain neurons suffice for circuit analysis — burny_tech · 2026-10-27
- EMNLP paper GENIE: a fine-grained metric locating where LLM stories are novel — gregd_nlp · 2026-10-27
- Hashed n-gram value embeddings push GPT to 3.2115 loss, beating 30M track record — RobertTLange · 2026-10-27