Three Weeks with Stanford CS329A: Generators Have Outrun Verifiers

James Le spent three weeks working through Stanford Online's publicly available Fall 2025 course CS329A "Self-Improving AI Agents," taught by Azalia Mirhoseini (ex-Google) and Aakanksha Chowdhery, and wrote a long recap of each of the nine lectures. The notes systematically frame the central tension of today's AI landscape: the gap between generators and verifiers.

Confirmed

• The course's core thesis is that generators have far outpaced verifiers, a thread running through all nine lectures.
• DeepSeek-Coder-V2-Instruct solves only 15.9% of SWE-bench Lite with a single sample but 56% after 250 samples: with a free verifier, massive sampling beats single-pass capability.
• DeepSeekMath-V2 treats verification as a product rather than a constraint, scaling verification compute alongside the generator; whether this works beyond math remains unknown, Le notes.
• Process reward models once required 800,000 human labels; Math-Shepherd replaces annotators with rollouts, lifting GSM8K to 84.1%.
• Lecture 4 is framed as a ladder of weaker signals: environment feedback (ReAct), test-suite feedback (RLEF), written-document feedback (Constitutional AI) — each cheaper and more ambiguous.
• Citing DeepSeekMath's own measurements, Le highlights the strongest challenge to the claim that RL with verifiable rewards teaches new capabilities: RL improves Maj@K but not Pass@K.
• AI measurement is splitting: METR puts o3's 50% time horizon at roughly 110 minutes of human work, doubling about every 7 months, yet no system exceeds 31% on DeepScholar-style literature-review benchmarks.

Why it matters

• If generation keeps outrunning verification, research focus shifts toward scaling verification compute, and verifiers may become a new competitive frontier and product form.
• The Maj@K-not-Pass@K data suggests RLVR may improve reliability of existing capabilities rather than create new ones, shaping expectations for the RLVR route.
• Doubling time horizons coexisting with low scores on concrete tasks like literature review shows single benchmarks no longer describe AI capability; the evaluation landscape itself is fragmenting.

2026-09-16 ~ 2026-09-16 · 9 related posts

• [source] Stanford CS329A publishes 9-lecture course on self-improving AI agents — le_james94 · 2026-09-16
• [source] Generator outruns verifier: 250 samples lift DeepSeek-Coder to 56% on SWE-bench — le_james94 · 2026-09-16
• Repeated sampling lifts DeepSeek-Coder on SWE-bench Lite from 15.9% to 56% — le_james94 · 2026-09-16
• Math-Shepherd replaces 800K human labels with rollouts, lifting GSM8K from 77.9% to 84.1% — le_james94 · 2026-09-16
• A ladder of weaker feedback signals: ReAct, RLEF, Constitutional AI — and Math-Shepherd's rollouts — le_james94 · 2026-09-16
• DeepSeekMath data: RL on verifiable rewards improves Maj@K but not Pass@K — le_james94 · 2026-09-16
• o3's time horizon hits ~110 minutes yet no system clears 31% on DeepScholar-Bench — le_james94 · 2026-09-16
• DeepSeekMath-V2 makes verification the product, scaling verifier compute ahead of the generator — le_james94 · 2026-09-16
• [source] 3 weeks through Stanford CS329A: the generator has outrun the verifier — le_james94 · 2026-09-16