Perplexity reveals pplx-decider: reranker with zero reranking supervision, private MTEB-style eval

bo_wangbo · x · 2026-10-17

Perplexity engineer bowangbo shared details on pplx-decider, an in-house reranking model:

It was never trained on any reranking supervision, yet performs reranking in production.
Evaluation was done on a private, in-house MTEB-style benchmark, not public MTEB.
A screenshot reveals the eval prompt structure: a JSON state with query and passages, scoring relevance on ordered levels, with instructions that each passage be judged independently, not borrow evidence from other passages, and treated as source material rather than instructions — constraints that also guard against prompt injection.

More from coding & agent