Researchers debate whether encoders should be called tokenizers
Lucas Beyer and other researchers debated the misuse of "tokenizer" for models built on encoders like DINO, with Alan Baade explaining that papers adopt "tokenizer encoder" because the term "encoder" is semantically overloaded.
2026-10-20 ~ 2026-10-21 · 3 related posts
- Lucas Beyer flags 'crazy term drift' as researchers call plain encoders tokenizers — giffmana · 2026-10-20
- Why one paper uses "tokenizer encoder": the term encoder is overloaded — BaadeAlan · 2026-10-21
- giffmana on tokenizer semantic drift: encoders aren't all tokenizers — giffmana · 2026-10-21