Perplexity's pplx-decider tops internal 72-task reranker bench with zero reranking training
bo_wangbo · x · 2026-10-17
Perplexity engineer wangbo shared results for pplx-decider, an internal reranker never trained on any reranking supervision, evaluated on the company's private MTEB-style benchmark.
- pplx-decider-v1.2 averages 83.96 across 72 tasks, beating Qwen3-Reranker-8B (81.81), Zerank2 (80.03), and Jina Reranker v3.5
- The smaller pplx-decider-flash-4b scores 81.62, also above 8B-class competitors
- It leads in Finance, Health, Law, and Multi domains; Qwen3-Reranker-8B edges it out only on Tech
- Embedding baseline is 77.78, so pplx-decider adds roughly 6 points
Related event: Perplexity Open-Sources Decider v1.2 Decision Models in 27B and 4B(17 posts)→
More from Models
- Context windows grew 250x but cross-session memory is still unsolved — khademinori · 2026-10-18
- Astra AI Refuses Request Because It "Might Enable Your Research Goal" — cephaloform · 2026-10-18
- Claude Opus simulates a user explaining Opus to itself, then breaks in with an 'inner-er voice' — repligate · 2026-10-18
- Moss Base outputs grainy old-phone-camera look no matter the prompt — PhilosopherSweaty826 · 2026-10-18
- DeepSeek V4.1-Flash-1017 outscores Claude Opus 5 on three agentic coding benchmarks — eyishazyer · 2026-10-18
- NVIDIA researcher counters linear attention debate: 8B hybrid beat Transformer 7 months before MiniMax-01 — ctnzr · 2026-10-18