Debate: one big model is offline RL, specialized models are online RL
willcb · x · 2026-11-17
In a discussion with swyx and PimDeWitte, willcb offers a framing: at the macro level, one big model equals offline RL while specialized models are online RL. He also argues that urging everyone to 'build their own RL envs and tune hyperparameters' — like telling devs to curate agent workspaces and install skills — is anti-AGI-pilled: both are training wheels for toddler AGI that model progress will render obsolete.
More from AGI Musings
- Sam Altman warns chipmakers are double-counting OpenAI demand, fueling overbuild risk — bookwormengr · 2026-11-17
- Domingos on fast takeoff: 'The world is a jungle, not a racetrack' — pmddomingos · 2026-11-17
- willcb: RLaaS is just a stepping stone toward continual learning, the real AGI bet — willcb · 2026-11-17
- Domingos: recursive self-improvement has existed since the 1950s — pmddomingos · 2026-11-17
- Black Friday will be the moment consumer agents go mainstream and botting becomes everyone's game — omooretweets · 2026-11-17
- AI safety researchers lament self-regulation trap: too soon in 2023, too late in 2026 — tegmark · 2026-11-17