giffmana on tokenizer semantic drift: encoders aren't all tokenizers
giffmana · x · 2026-10-21
Researcher giffmana (of MAE and DINO fame) weighed in on X about the shifting meaning of "tokenizer": an encoder is any component that turns raw input into a more usable form — MAE's encoder, DINO's encoder, SigLIP's image/text encoders all qualify — but only discrete-codebook encoders like VQ-VAE's are truly tokenizers. He notes the LLM-era generation is slowly drifting the term, joking he's "old man yelling at cloud." Quoted Alan Baade explains the paper used "tokenizer encoder" because "encoder" alone is too overloaded across enc-dec Transformers and MAE/MAR contexts.
Related event: Researchers debate whether encoders should be called tokenizers(3 posts)→
More from Research
- Stanford's LILIE loss cuts text-to-image gFD by 14 points over pixel-only training — StanfordUniversity · 2026-10-21
- ETH's HumanoidPose recovers full-body robot pose from plain video, no internal sensors — ETHZurich · 2026-10-21
- One L-system rule grows a tree, fern, lightning, river and lungs in a 45s code film — prasenx · 2026-10-21
- MOSS-Image's text encoder is multimodal and accepts image inputs — RandumbRedditor1000 · 2026-10-21
- Skip the open access fees: NIH-funded papers must go on PMC with no embargo — anshulkundaje · 2026-10-21
- Creed: LLM code auditing infers developer beliefs to find semantic security flaws — mboehme_ · 2026-10-21