Why LLMs can innovate: The role of search and verification
bindureddy · x · 2026-08-31
Core Thesis: LLMs invent new things not because answers are in the training data, but because they master operators and patterns. They innovate by performing massive test-time search, composing known elements, and using verifiers to filter for truth.
Technical Logic:
- Proposal vs. Verification: Proposing solutions is hard, but verifying correctness (e.g., using Lean for math proofs) is cheap.
- Search Space + Verifier: In domains with known rules (biology, protein folding, materials), LLMs can explore vast search spaces and rely on verifiers to find valid results.
Examples:
- AlphaProof: Used Lean as a verifier to search the proof space, winning IMO silver.
- FunSearch: Combined LLM proposals with a checker to discover genuinely new math results for the cap set problem.
- AlphaFold: Generalized to uncrystallized proteins by learning the rules of folding.
Implication: Fields like biology and energy fit the "massive search space + cheap verification" pattern. Once AI learns the universe's patterns, cracking cancer or infinite energy becomes theoretically possible.
Related event: How LLMs Invent New Things: Search and Verification(4 posts)→
More from Research
- A beginner-friendly introduction to causal inference kicks off, Part 1 — Nasereliver · 2026-10-16
- FBI method trains for Pass@K with one sample per update at 12.3% of the cost — EmtiyazKhan · 2026-10-16
- Berkeley researchers sample humanoid robot policies directly from weight space with diffusion — berkeley_ai · 2026-10-16
- OneWM: unified causal training shows world and action modeling improve each other — IAAI233 · 2026-10-16
- Baidu's Virtual Head Attention tames the KV cache, FLOPs, quality triangle in long-context LLMs — baidu · 2026-10-16
- 5.8M-param Speech Event Detector adds laughs, breaths and背景 sounds to Whisper transcripts — Ayush Kumar Bar · 2026-10-16