OpenAI Tried New Architectures Only 3-4 Times in 7 Years, Says Jerry Tworek
andrew_n_carr · x · 2026-08-26
Jerry Tworek, a seven-year OpenAI veteran, reveals the company attempted new architectures only three or four times in that span — and the process was brutal:
- A researcher first had to code a small-scale experiment, which was itself difficult;
- Success then required at least three months of validation;
- Scaling up meant persuading 10 people to back the idea, followed by another 3-6 months of scaling work;
- Most ideas either lost to the momentum of the already-working Transformer, or got partially absorbed and faded away.
The commenter quips: this is either fully "Bitter Lesson"-pilled or not nearly pilled enough — only time will tell.
More from Companies & People
- Musk floats Amazon rival after retailer blocks Grok shopping bots — robleclerc · 2026-10-15
- Ex-White House cyber deputy launches SAIF nonprofit with $25M from a16z founders for AI cyber defense — vishalmisra · 2026-10-15
- YC F26's GodHands launches: AI learns company know-how by watching people work — ycombinator · 2026-10-15
- Ex-Apple engineer: one internal iOS app took Siri QA from 50 to 300 bug reports a day — jdluk87 · 2026-10-15
- Staff-level engineers get poached and offered part-time gigs — but must deliver immediately — teodorio · 2026-10-15
- AI film and TV hit the market: Prime Video picks up AI spin-off, pilots screen at Mipcom — lmoroney · 2026-10-15