Embedding researcher: look beyond BEIR — mixedbread and Perplexity's decider clearly lead
bclavie · x · 2026-10-27
Embedding researcher bclavie cautions against evaluating retrieval models on BEIR alone: on better evaluation suites, mixedbread and Perplexity's decider are noticeably further ahead than the rest of the list. He also teases that more is coming.
More from Models
- Opus 5.5 keeps getting blocked as [cyber] when writing SASS code for GPUs — DimitrisPapail · 2026-10-27
- Prague lab BottleCapAI releases ThinkingCap models cutting thinking tokens up to 69% with negligible accuracy loss — JFPuget · 2026-10-27
- Recursive SI trained a model from scratch in 4 months using auto research — altryne · 2026-10-27
- New AI fingerprint in support tickets: "please do not upgrade our plan" sign-offs — glcst · 2026-10-27
- 36 blind AI judges pick a solo-session agent setup over Superpowers 36-0 — Kai_ThoughtArchitect · 2026-10-27
- New benchmark shows how often AI models hallucinate nonexistent skin conditions — Individual-War3274 · 2026-10-27