Perplexity's decider model matches top reranker at $0.20/1k queries, cutting cost 80%
antoine_chaffin · x · 2026-10-27
A hev mind benchmark shows Perplexity's pplx-decider-v1.2 now matches Mixedbread v3.1, last month's best listwise reranker, at 0.511 avg nDCG@10 over SciFact/NFCorpus/FiQA (paired diff +0.0001, statistically indistinguishable) while charging $0.20 per 1,000 queries. Mixedbread also cut prices to roughly $0.27/1k. Top-tier reranking costs fell about 80% in a month as a general decision model caught up with the specialist. The author cautions BEIR wins don't guarantee all-domain performance and suggests tuning state definitions and document packing to squeeze more.
More from Models
- Yacine: Open-source LLMs must share training data, weights alone aren't enough — yacineMTB · 2026-10-27
- Switzerland's fully open Apertus 1.5 adds multimodal input, reasoning, longer context — yacineMTB · 2026-10-27
- Temperature extremes make Google's model switch from refusal to 'universe destruction guide' — Gauri_the_great · 2026-10-27
- Effort-level pricing tiers dug up before the official switch ships — PawelHuryn · 2026-10-27
- Bug Hunt Bench: 105 real bugs show GPT-6 scoring 3-5x higher than rival models — PawelHuryn · 2026-10-27
- Open-source RP model Flint: GRPO with LLM judge lifts writing score 4.92→6.90 — Mysterious_Hearing14 · 2026-10-27