Perplexity's 4B decider model distills from its 27B sibling using the same training recipe
antoine_chaffin · x · 2026-10-17
antoinechaffin details the 4B decider model: it shares all design decisions with the 27B — attention schema and training recipe — with the only difference being that it learns from the larger model's signal (distillation), making it exceptionally strong at its scale.
More from Models
- Claude cracks 30-year mystery: ECDSA curve seeds were just RNG output — scaling01 · 2026-10-17
- KOL teases mystery model argon, says people are 'severely underestimating' it — iruletheworldmo · 2026-10-17
- Reddit post argues Anthropic has led benchmarks for 6+ months and is 2-3 iterations ahead internally — Concern-Excellent · 2026-10-17
- Perplexity traces decider v1.2 regressions to longer training, mitigates via model souping — antoine_chaffin · 2026-10-17
- Dual-mask fine-tuning recovers cacheability with only -0.4 DI vs full bidirectional — antoine_chaffin · 2026-10-17
- Perplexity open-sources next-gen decider models in 27B and 4B sizes with up to 5x faster inference — antoine_chaffin · 2026-10-17