Karpathy on Context Windows vs Fine-Tuning
No Priors · youtube · 2026-07-14
Andrej Karpathy believes that a context window is a "cheap" way to manipulate AI behavior, but it is fundamentally different from genuinely fine-tuning or customizing a model while trying to preserve its original capabilities.
He emphasizes that the science of how to "manipulate the brain" isn't fully researched yet. Compared to simply stuffing information into the context, stably modifying a model's capabilities is far more complex.
More from AGI Musings
- OpenAI researcher: space operas now need ambiguously aligned superintelligences for realism — jachiam0 · 2026-09-11
- MD shows buying lab media requires background checks, calling AI bioweapon doom scenarios implausible — Ghost_Pilot_MD · 2026-09-11
- AI researcher: AI killing humanity on its own is sci-fi; real risk is misuse by people — JFPuget · 2026-09-11
- Only a 4-day window: timeline casts doubt on OpenAI's independent math result claim — gleech · 2026-09-11
- Hesamation: 50,000 OpenAI agents may be burning millions overnight on P vs NP and Riemann hypothesis — Hesamation · 2026-09-11
- Dev argues AI safety status quo isn't safe: aging kills everyone within ~120 years anyway — tomchapin · 2026-09-11