Project APE says verifier costs fell about 90x in a year as Chinese open models lead
soumitrashukla9 · x · 2026-07-22
A second result from the same verifier benchmark says the cost of verification is collapsing.
- The cheapest cost for 50% recall fell by roughly 90× over the past year.
- Costs at lower recall fell even more.
- The authors note that open-weight Chinese models are a major part of the current cost-reliability Pareto frontier.
If the trend continues, the paper argues that verifying thousands of papers could eventually become close to free.
More from Research
- Stanford's PaperFormalizer turns research papers into Lean-verified versions with author in the loop — StanfordUniversity · 2026-10-27
- JockeyBench uses Japanese horse racing to test LLM agents — all betting strategies lose money — Zhongwei Xie · 2026-10-27
- Cohere's Invent-A-Dataset paper: zero-seed synthetic data beats frontier APIs on quality and diversity — idanbeck · 2026-10-27
- Hugging Face TRL makes score centering + masked importance sampling default in async GRPO — _lewtun · 2026-10-27
- AutoResearch agent's weird training curve gets dug into instead of discarded — ZimingLiu11 · 2026-10-27
- EMNLP 2026: sub-billion-parameter LMs show superhuman entity tracking, human-like fleeting memory — GretaTuckute · 2026-10-27