OpenAI Tried New Architectures Only 3-4 Times in 7 Years, Says Jerry Tworek
andrew_n_carr · x · 2026-08-26
Jerry Tworek, a seven-year OpenAI veteran, reveals the company attempted new architectures only three or four times in that span — and the process was brutal:
- A researcher first had to code a small-scale experiment, which was itself difficult;
- Success then required at least three months of validation;
- Scaling up meant persuading 10 people to back the idea, followed by another 3-6 months of scaling work;
- Most ideas either lost to the momentum of the already-working Transformer, or got partially absorbed and faded away.
The commenter quips: this is either fully "Bitter Lesson"-pilled or not nearly pilled enough — only time will tell.
More from Companies & People
- Critique: Academia's low standards hinder AI response — RexDouglass · 2026-08-26
- Talent Strategy: Bet Early on Niche Experts or High-Slope Learners — benaratame · 2026-08-26
- Dev Battlefield Shifts to Discovery, Trust, and Bundling — tinyfool · 2026-08-26
- Enterprises ditch Fable 5 as cheaper open-source models win on value — FinanceYF5 · 2026-08-26
- Era of US closed-model supremacy and premium pricing is ending — brucemacv · 2026-08-26
- Semiconductor Veterans Launch C2i to Revolutionize Power Delivery for AI — santoshpanda · 2026-08-26