Researchers call encoder/decoder Transformer naming a misnomer: it's all attention masks

Researchers argue the encoder/decoder distinction in Transformers is misleading: everything is just attention plus masks, and "decoder-only" models were never actually decoders.

2026-09-10 ~ 2026-09-10 · 2 related posts

• Decoder-Only Transformers Were Never Decoders: Researcher Calls Out Terminology Mix-up — TimDarcet · 2026-09-10
• Researchers argue Transformer encoder/decoder naming is just attention masks — TimDarcet · 2026-09-10