Survey of 119 papers maps benchmark compression: estimating full LLM leaderboard scores from partial evaluations
SunYatsen · hf · 2026-10-15
Researchers at Sun Yat-sen University published the first systematic survey of benchmark compression — estimating full-benchmark scores and rankings from partial LLM evaluations to cut testing costs.
- Reviews 119 papers from a field that has grown rapidly since 2024.
- Proposes a taxonomy: compression within single benchmarks, across benchmark suites, and during adaptive evaluation, plus work on redundancy and reliability.
- Methods cut costs via representative subsets, cheaper/reusable information, and adaptive testing, but risk poor coverage, inaccurate estimates, and sequential delays.
- Includes a practical guide for choosing, validating, and deploying these methods.
More from Research
- Training the top 0.1% coordinates by Fisher info recovers ~85% of full PG gains — YouJiacheng · 2026-10-15
- How did NLP researchers cope when LLMs solved their field? A lesson for what's next — iScienceLuvr · 2026-10-15
- Alibaba's HICO-Bench shows 36% of highly-rated agent conversations fail verification — Tongyi-ConvAI · 2026-10-15
- PixText fixes text in generated images: 83% glyph pass rate vs 37% for baselines — ATH-MaaS · 2026-10-15
- TextVerifier catches flawed Chinese glyph rendering with F1 0.844, far above OCR and VLMs — ATH-MaaS · 2026-10-15
- ByteDance's Diff Diffusion predicts code edit paths: 2.78x faster inference, 39% lower latency — Xiaolong Xu · 2026-10-15