Blog argues pretraining data, not verifiability, makes LLMs good at math and coding
_arohan_ · x · 2026-09-19
A blog post makes a contrarian argument: LLMs excel at math and coding not because of verifiability, but because of pretraining data distribution — math and code on the internet are simply exceptionally learnable text. The piece challenges the common "verifiable rewards explain it" narrative and is worth reading alongside RLVR debates.
More from Research
- TMLR swamped by submissions, quizzes 10 authors to prove papers are their own — ylecun · 2026-09-19
- Models press a button less often when it removes an injected steering vector — MoonL88537 · 2026-09-19
- OpenADMET lands Gates Foundation funding to build open toxicity data for drug discovery — anshulkundaje · 2026-09-19
- Ken Goldberg's Agentic Robotics: AI writes deterministic robot code, no demos needed — animesh_garg · 2026-09-19
- Researchers question antibody model's SabDab training data extending to 2025, flagging test contamination — anshulkundaje · 2026-09-19
- Yale PhD quit after lymphoma diagnosis to build foundation models for antibody drugs at Aureka Bio — anshulkundaje · 2026-09-19