MIT Tech Review Explores the Next Era of LLMs: Startups Challenging the Transformer
volokuleshov · x · 2026-08-11
MIT Technology Review highlights that while the Transformer architecture currently dominates the AI industry, it is showing its age, particularly regarding the high computational cost of processing long contexts. Recent LLM advances, such as reasoning capabilities and massive context windows, are often workarounds patching fundamental flaws rather than neat extensions.
A wave of startups is now attempting to break free from the Transformer paradigm to develop next-generation models, dubbed LLMs+. The article spotlights companies like Subquadratic exploring sub-quadratic architectures, alongside other cutting-edge directions like diffusion language models (e.g., Mercury 2), aiming to fundamentally redefine how large language models are built.
Related event: MIT Tech Review Explores Alternatives to Transformer(2 posts)→
More from Companies & People
- OpenAI's Heads of Ethics, Safety Systems, and Mission Alignment Have All Quit — ns123abc · 2026-08-11
- Coinbase, Shopify, Ramp Build Custom Coding Agents but Still Pay Anthropic — Familiar_Jaguar_4134 · 2026-08-11
- OpenAI's Head of Ethics Leaves the Company Less Than a Year After Joining — Polymarket · 2026-08-11
- Meta Unveils New Open Models as Zuckerberg Outlines AI Strategy in 6,000-Word Essay — Ars Technica AI · 2026-08-11
- OpenCode Is Best Positioned to Win the Multi-Model AI App Race — intellectronica · 2026-08-11
- Prominent Scholar to Google: Stop Reducing DeepMind to Just an LLM Lab — pmddomingos · 2026-08-11