The unreasonable effectiveness of BM25 for agentic search — Jo Kristian Bergum, Hornet.dev
AI Engineer · youtube · 2026-09-16
Search veteran Jo Kristian Bergum (Hornet.dev) argues on the AI Engineer podcast that BM25 — the 30-year-old lexical scoring function ("Best Match 25", named after the 25th scoring experiment) — is making a comeback in the agent era without changing at all. What changed is the user: models write far longer, more specific queries than humans and can fire off a dozen at once.
Key points:
- A deep-research benchmark of 830 riddle-like questions over 100k web documents shows stuffing answer-bearing documents into context yields high accuracy even for older models — reasoning was never the bottleneck; handing the same model a search tool drops accuracy, since it now depends on query formulation and the retriever.
- Bergum likens context windows to floppy discs (350k tokens before quality degrades), so something must still decide what goes in.
- He favors a pattern of dumping retrieved documents into a file-system workspace and letting the model use grep and other primitives it was already trained on.
- Timestamps cover BM25 naming history, model query formulation, classical evaluation being dead, and four takeaways.
More from coding & agent
- Microsoft's MemSteer turns unlabeled user histories into agent memory supervision — MicrosoftResearch · 2026-11-02
- She fired herself from her own Instagram: AI agent runs the account, views up 82.3% — heyshrutimishra · 2026-11-02
- Sourcegraph: AI writes code faster than teams can review it, driving more outages — AI Engineer · 2026-11-02
- Tsinghua & Microsoft's HarnessMM cuts agent token use by up to 87.93% while raising success rate — rohanpaul_ai · 2026-11-02
- GitHub launches Agent HQ: Anthropic, OpenAI, Google coding agents in Copilot — emmanuelvivier · 2026-11-02
- DeepSeek open-sources Super-Agent, a coding agent that reads team chats to minimize human input — emmanuelvivier · 2026-11-02