Perplexity decider v1.2 lands #3 on Decision Index as longer training erodes backbone skills
antoine_chaffin · x · 2026-10-17
antoinechaffin shares decider v1.2 results: the 27B scores 67.7 public / 64.0 same-skills (Full 65.5, rank #3, 144ms median latency), behind new DI leader StarLux; the 4B leads its scale at 54.8.
- Private/public gap slightly wider than last gen but expected — in-domain gains, while same-skills parity with D3 shows real task-level generalization, not benchmark overfitting
- New-domains score fell below D3: the authors concede longer training drifted the model from its backbone capabilities, to be fixed in future releases
- Vision results unpublished but expected strong despite almost no in-domain vision training data
More from Models
- Claude cracks 30-year mystery: ECDSA curve seeds were just RNG output — scaling01 · 2026-10-17
- KOL teases mystery model argon, says people are 'severely underestimating' it — iruletheworldmo · 2026-10-17
- Reddit post argues Anthropic has led benchmarks for 6+ months and is 2-3 iterations ahead internally — Concern-Excellent · 2026-10-17
- Perplexity traces decider v1.2 regressions to longer training, mitigates via model souping — antoine_chaffin · 2026-10-17
- Dual-mask fine-tuning recovers cacheability with only -0.4 DI vs full bidirectional — antoine_chaffin · 2026-10-17
- Perplexity open-sources next-gen decider models in 27B and 4B sizes with up to 5x faster inference — antoine_chaffin · 2026-10-17