Transformers are samplers: verifiers can't fix unreliable code generation
gerardsans · x · 2026-10-23
- In a debate with @callebtc, gerardsans argues that transformers are fundamentally samplers: of N generated samples, k fail, and harnesses retry on your behalf, adding considerable overhead in code generation.
- The pipeline of sample-verify-repeat keeps the same underlying failure rate; formalisation doesn't reduce errors.
- His key claim: improving verifiers doesn't address the most pressing problem — the unreliability of model outputs itself.
Related event: Debate: Transformers are samplers, limiting code reliability(3 posts)→
More from coding & agent
- MIT Sloan research: AI speeds up coding but delivery still bottlenecks on testing — sven_ai · 2026-10-23
- Agents logging into bank accounts violate most banks' terms, even with user consent — pstAsiatech · 2026-10-23
- Codex desktop shipped with no editor: 4-person team bet everything on agents — sujingshen · 2026-10-23
- Under matched token budgets, most gains of web agent online augmentation vanish — hllo_wrld · 2026-10-23
- First company officially incorporated by an AI agent via Stripe's Atlas MCP — jeff_weinstein · 2026-10-23
- Designing a full motion system in one weekend with AI, in low-effort mode — soleio · 2026-10-23