Team Benchmarks Every SOTA Cross-Encoder on 70+ Tasks: Zero Average Gain
bo_wangbo · x · 2026-10-09
In a discussion on rerankers, bowangbo reports his team benchmarked every "SOTA" cross-encoder from 200M to 8B parameters, point-wise to list-wise, across 70+ tasks in a wide range of domains. Result: after averaging gains and losses, ALL cross-encoders deliver zero gain. He claims an alternative consistently wins without any score supervision or contrastive training, though he declines to disclose cost. The thread also raises benchmark contamination concerns around BEIR.
Related event: 70+ Task Benchmark Shows SOTA Cross-Encoders Yield Zero Average Gain(2 posts)→
More from Research
- RFD4-Proteina's 'underrated' docs detail partial diffusion for protein redesign — AllThingsApx · 2026-10-23
- MPI-Sec PhD Thesis on Vulnerability Detection Benchmarks Includes Pre-LLM Classic "Top Score on the Wrong Exam" — mboehme_ · 2026-10-23
- LiveNerf's 29-day data finds no nerfing of Claude Opus 5.5 — TheOnlyVibemaster · 2026-10-23
- New paper makes 9 falsifiable predictions toward a major AI-enabled terror attack by 2027-2028 — StephenLCasper · 2026-10-23
- HumanoidPose recovers humanoid robots' full-body joint states from RGB video alone — cholz · 2026-10-23
- Sophont's Holon-S1 predicts genes and biomarkers from routine pathology slides — iScienceLuvr · 2026-10-23