OpenAI Tried New Architectures Only 3-4 Times in 7 Years, Says Jerry Tworek
andrew_n_carr · x · 2026-08-26
Jerry Tworek, a seven-year OpenAI veteran, reveals the company attempted new architectures only three or four times in that span — and the process was brutal:
- A researcher first had to code a small-scale experiment, which was itself difficult;
- Success then required at least three months of validation;
- Scaling up meant persuading 10 people to back the idea, followed by another 3-6 months of scaling work;
- Most ideas either lost to the momentum of the already-working Transformer, or got partially absorbed and faded away.
The commenter quips: this is either fully "Bitter Lesson"-pilled or not nearly pilled enough — only time will tell.
More from Companies & People
- DeepSeek outage compensation reportedly hands single accounts over $850K in API credits — teortaxesTex · 2026-10-15
- Jensen Huang's pep talk message to the UK's AI sector goes viral — threepointone · 2026-10-15
- CyberiadLabs partners with Hyver to offer free AI courses to 7,830 students across 6 African countries — jbarbier · 2026-10-15
- Anthropic slowdown could trigger a domino effect across the AI startup ecosystem — AccBalanced · 2026-10-15
- A 6-person team shipped healthcare prior-auth AI agents in 3 months — here's the playbook — alex_verem · 2026-10-15
- Scale AI exec: data companies profit because frontier labs don't own the product surface — sarahookr · 2026-10-15