willcb declares transformers done, says future belongs to 'freaky-looking sorta-transformers'
willcb · x · 2026-09-10
Developer willcb posted a pointed take: "transformers are done. the future is freaky-looking sorta-transformers." The one-liner captures a growing sentiment among practitioners that pure transformer architectures are giving way to hybrid variants — linear attention, state-space mixes and other modified designs — even though the post offers no supporting detail.
Related event: Developer Declares Transformers Dead, Future Belongs to Weird Variants(2 posts)→
More from Models
- DeepSeek's V4.1 Flash reworks the Transformer again: asymmetric encoder-decoder and CSA2 — evijit · 2026-09-10
- DeepSeek returns to encoder-decoder with new open architecture changes — evijit · 2026-09-10
- 100M output tokens for $60: DeepSeek off-peak pricing undercuts Opus 5 by 40x — airesearch12 · 2026-09-10
- Bug Hunt Bench ranks frontier coding models on 105 real planted bugs, with 200x cost spread — PawelHuryn · 2026-09-10
- Dev on both Claude and ChatGPT $20 plans hits limits, calls pricing convoluted — MaxLenormand · 2026-09-10
- Rumor: DeepSeek trained a 3T-parameter model but held back over serving economics — zephyr_z9 · 2026-09-10