ARBITER: 35B MoE Verifier Pushes GPT-OSS-120B Past IOI Gold-Medal Threshold
Zheye Deng · hf · 2026-10-15
Agentic Code Judging with Verifiable Feedback: ARBITER
Problem: competitive-programming judges return verdicts without failure explanations, yet test-time scaling needs verifiers that can diagnose candidate errors.
Approach
- Build USACO-JUDGE from USACO 2026 contests, using accepted/unaccepted programs from human experts and LLMs
- Task: judge and diagnose a program from the problem statement and source alone, with generated counterexamples as checkable evidence
- Train ARBITER, a 35B MoE agentic verifier, with RL in a generic execution sandbox
Results
- Counterexample success on incorrect programs: 40.61% → 62.67%
- Binary verdict accuracy: 71.41% → 87.41%, near frontier verification performance
- Beats baselines at code selection and refinement; in an IOI 2026 case study, ARBITER-guided refinement lifts GPT-OSS-120B's expected score above the gold-medal threshold under the official submission limit
More from Research
- New paper: Image Tokenizers Should Be Trained Between Images cuts artifacts, speeds training — ostrisai · 2026-10-15
- Robocurve releases all 2,109 robot runs with transcripts and videos; Gemini 4 Argon averages 33.8% — zacharynado · 2026-10-15
- Schmidhuber renews residual connections credit claim: "LSTM is a ResNet rotated 90 degrees" — SchmidhuberAI · 2026-10-15
- PPO Collapses LLM Output Distributions While GFlowNet Objectives Keep Them Uniform — kastnerkyle · 2026-10-15
- Harmonic representations in symbolic music transformers are localized and controllable — kastnerkyle · 2026-10-15
- Class Weights 101: The One-Line Fix for Imbalanced Data That Beats SMOTE — dima806_dima · 2026-10-15