The Evolution From Transformer to ChatGPT
stanfordnlp · x · 2026-07-20
This thread reviews the evolution of large language models: - In 2017, Transformer emerged, clearly outperforming RNN/LSTM. - In 2018, BERT gained popularity, with Google driving the industry to train, distill, and modify BERT; meanwhile, OpenAI persisted with the decoder-only GPT route. - In 2019, BERT profoundly impacted search, and Google released T5; in 2020, GPT-3 exploded in the community, causing a sensation with GitHub trending, public video demos, and SQuAD performance. - In 2021, Stanford proposed the foundation model concept, attempting to unify the previous Transformer/BERT/GPT narratives. - In 2022, ChatGPT was described as the productized result of pairing GPT-3.5 Turbo with a frontend and infrastructure. The core takeaway: while the public often views ChatGPT as a sudden breakthrough, it was actually the culmination of years of model evolution from a research and product roadmap perspective.
More from Models
- Reddit asks whether Kimi K3 is already good enough for production agents — CommercialClient2408 · 2026-07-21
- Korean startup says its model scored 44 on AAII and matches DeepSeek V4 Pro — JungWooHa2 · 2026-07-21
- OpenAI’s GPT-6 is predicted to be far more efficient than Fable — bindureddy · 2026-07-21
- Moonshot spotlights Kimi K3 and its API platform — pstAsiatech · 2026-07-21
- Motif 3 Beta lands on Hugging Face as South Korea’s foundation-model race heats up — Secure_Smoke_4280 · 2026-07-21
- Sakana AI’s Fugu-Cyber update tops real-world security benchmarks — SakanaAILabs · 2026-07-21