Frontier pretraining researcher: steering AI research still demands precise human direction
menhguin · x · 2026-10-07
A developer working on text-to-weights (sparse MoE pretraining) shares that current models still depend heavily on human research direction: he had to deliberately steer the model's ideas and articulate goals in very precise terms, or it would produce a dozen half-plausible suggestions that don't help. He also notes the absence of slop AI pretraining papers in recent years — likely because pretraining is just high-effort enough to deter slop factories. AI can mechanically pump out high-quality research, but human discretion remains essential.
More from AGI Musings
- Researcher predicts AI will solve every Millennium Problem and absorb all of mathematics — basedjensen · 2026-10-07
- Ofir Press imagines coding breakthroughs: doing what people thought would take 100 years — OfirPress · 2026-10-07
- 'Solve real problems' critique of OpenAI math models gets a sharp rebuttal — basedjensen · 2026-10-07
- 'Wake me up when AI cures cancer' — why dismissive takes on AI progress miss the point — basedjensen · 2026-10-07
- Are programming languages suboptimal for agents? Devs debate a coming compiler/OS rethink — peterjliu · 2026-10-07
- Yacine: posttraining is just the rich man's inference — yacineMTB · 2026-10-07