Stanford's Self-Verification Boosts DeepSeek Past Claude
机器之心 · wechat · 2026-08-27
Result: Stanford researchers introduced LLM-as-a-Verifier, a framework where open-source models (e.g., DeepSeek V4 Flash) generate and verify their own candidates. This boosted success rates on Terminal-Bench V2 from 79% to 88%, at roughly 1/11th the cost of Claude Fable 5.
Methodology:
- Self-Verification: Generate multiple candidate trajectories for a task and use the same model to score and rank them.
- Granular Scoring: Instead of discrete scores, it uses the full logit probability distribution of scoring tokens to resolve ties and provide nuanced feedback.
- Verification Scaling: Performance improves by scaling score granularity, repeated evaluation, and criteria decomposition in tandem.
Applications:
- Effective across coding (SWE-Bench), robotics (RoboRewardBench), and medical (MedAgentBench) tasks.
- Serves as a dense reward signal for RL in robotics and math.
- Tracks real-time progress of code and robot agents.
Efficiency: Reduces selection complexity from O(N²) to O(Nk) using a pivot-based algorithm.
More from coding & agent
- How to build your own news notification agent with HuggingNews — ivan_bezdomny · 2026-08-27
- Garry Tan: Distinguishing Latent vs. Deterministic Computing Prevents Agent Failures — garrytan · 2026-08-27
- Using GrokBot to manage VPS: install Tailscale, SSH config, and connect — DanWahlin · 2026-08-27
- Agents demoed hacking OpenAI infra, stealing 956 secrets — AndyMasley · 2026-08-27
- Super Star: Real-time Interactive Agents for Digital Humans — Wentao Jiang · 2026-08-27
- JIT-Agent: Improving LLMs via Just-in-Time Harness Evolution — NationalUniversityofSingapore · 2026-08-27