After RL training, calling LLMs 'language predictors' is no longer accurate, researcher argues
morqon · x · 2026-09-25
Chris Hayduk argues that since a large share of training compute is no longer spent optimizing next-token prediction, characterizing LLMs as "predicting language" is a mischaracterization — at least once models have undergone substantial RL training.
If the phrase means anything, he says, it should mean "predicting the chain of language that will solve this difficult problem" rather than "predicting the next token" or "being a stochastic parrot." A direct challenge to the popular stochastic-parrot framing of post-RL models.
More from Models
- GPT-6 Astra beats NetHack on third try, first recorded LLM agent ascension — emollick · 2026-09-25
- AI models ran a vending machine business for a year: GPT-6 Sol turned $500 into $14,428 — 141_1337 · 2026-09-25
- Kevin Roose hands an AI agent $100 and a Kalshi account to test frontier models — MickeySteamboat · 2026-09-25
- Ex-Meta engineer benchmarks Jev vs GPT-nano: same score, 5x faster, 20% pricier — danielmckinn0n · 2026-09-25
- Altimeter CEO: OpenAI's Navier–Stokes-solving model withheld amid safety and govt scrutiny — rohanpaul_ai · 2026-09-25
- GPT-6 Sol lands on MathArena just behind GPT-6 Astra at lower cost — sandersted · 2026-09-25