A Better, Fully Open Successor to BEIR for Retrieval Eval Is Coming
bclavie · x · 2026-10-22
bclavie (of the co-contraction work) teases a new retrieval evaluation benchmark positioned as a much better and more discriminative replacement for BEIR and NanoBEIR, following his earlier claim that "BEIR is considered harmful." He says it will likely be fully open — worth watching for embedding/retrieval model teams.
More from Models
- Creator clarifies: Opus 5.5 brought existing city builder Microlandia into Minecraft, not built it — Angaisb_ · 2026-10-22
- 0.9B Model Transcribes a 4-Person Meeting on a MacBook With 0.08% Speaker Confusion — Binary_orchid · 2026-10-22
- Fireworks trains Meta's Muse Glimmer on Kalshi outcomes: +11.2pp accuracy over Jev — sophiamyang · 2026-10-22
- Gemini CLI PR reveals unreleased gemini-4-argon model with 1M context window — mr8lu · 2026-10-22
- Does php2shell really require top-tier cyber model capabilities? — xeophon · 2026-10-22
- Audit of 2,979 Gemini launch-week posts: 91.6% of complaints lack evidence — riverdancemcqueen · 2026-10-22