Training Is Mostly Test-Time Compute: Sampling Trajectories Beats Backprop
yunta_tsai · x · 2026-10-15
Researcher yuntatsai argues that most training is actually test-time compute: backpropagation is only a fraction of the process, while the majority consists of sampling as many trajectories as budget allows and collecting feedback—ideally new, high-value information worth the joules. In these asymmetric settings, self-improvement means scaling the test-time compute that yields the most useful information per joule.
More from AGI Musings
- Intelligence is proportional to data: reorder tasks by data volume and the jagged frontier becomes a spiral — sebkrier · 2026-10-16
- "Vibecoding" is as misleading a label as "AI" and "SI," dev complains — eschadiol · 2026-10-16
- Reviewer will decline papers largely written by AI, citing author effort — IanArawjo · 2026-10-16
- AI-written cold emails hit a wall: Pragmatic Engineer's reply gets templated AI response — marktenenholtz · 2026-10-16
- Ben Bajarin: Enterprise Agentic Adoption Is Infant-Stage, Spend Could Top $1T a Year — BenBajarin · 2026-10-16
- UMD Philosophy hires: junior Philosophy of Mind and joint AI Ethics/Safety positions — sethlazar · 2026-10-16