Same Material, Better Order: Reranking Lifts Correct-Passage Hit Rate to 90%
mikegiannulis · x · 2026-09-18
The thread reveals the test methodology: pulling real inputs from their own database, comparing against past Claude decisions and human labels or deterministic ground truth, and setting pass/fail rules before seeing results — a pure offline replay, no live customer experiment.
Headline result: for book draft source retrieval, embedding similarity ranked the correct passage first 60% of the time; Jev reranking hit 90%. Same material, better order — a much better starting point for writing.
More from coding & agent
- 15 harnesses, same result: informal test concludes agent harnesses don't matter — airesearch12 · 2026-09-18
- 4 deployment strategies explained via 4 visuals: feature toggle, blue-green, canary — _jaydeepkarale · 2026-09-18
- ServerKit: open-source server panel for apps, DBs and Docker hits 1.3k stars — tom_doerr · 2026-09-18
- Gary Marcus: Agents hold production credentials in a security gap nobody owns — GaryMarcus · 2026-09-18
- Amp's design philosophy: simple primitives, no tricks, let models improve it — HankYeomans · 2026-09-18
- You.com wraps AI Agentic Hackathon with self-improving agents challenge — PolarBearby · 2026-09-18