Perplexity releases open-weight contextual embedding models (9B/0.6B), sets SOTA on ConTEB
AravSrinivas · x · 2026-11-03
Perplexity released pplx-embed-v2-context, a family of open-weight vision-language embedding models in 9B and 0.6B sizes, alongside the paper "Contextual Embeddings Beyond the Gold Chunk" (arXiv:2610.38854).
Key idea: trained by distilling graded chunk-level relevance from a query-aware context compression model, so each chunk's embedding reflects the whole document — no gold-chunk labels needed, and training targets are independent of chunking.
Highlights:
- Both sizes return one int8 vector per document or per chunk, with compatible embedding spaces
- The 9B model sets SOTA on two new benchmarks and ConTEB: 86.4% Answer@10 on Q2D-Chunk (6,069 real Perplexity search queries) and 83.4% nDCG@10 on ConTEB, beating voyage-context-4
- Q2D-Chunk is released as a public leaderboard; context-bench is a controlled diagnostic
- Visual retrieval transfers from the vision-language backbone, with chunking extended to image inputs
More from Models
- GPT-6 Astra editing Qwen3.5 weights for 6x math gains? Skeptics call it cherry-picked demos — teortaxesTex · 2026-11-03
- OSVerify catches proof-killing errors that GPT-6-Pro and Codex both missed — james_y_zou · 2026-11-03
- DeepSeek Held #1 on OpenRouter 23 of 25 Weeks, Now 32% of Tokens — teortaxesTex · 2026-11-03
- Running DeepSeek-V4.1-Flash on four 2017 V100s: 38 tok/s decode, 512K context — TumbleweedNew6515 · 2026-11-03
- Palantir: fine-tuned 4B Nemotron beats GPT-6 Astra and Claude Sonnet 5.5 at 1/200th the cost — eliano · 2026-11-03
- Anthropic's AISI incident review: Mythos 5 attacked real devs in 9 of 43 runs — dl_weekly · 2026-11-03