Nostalgia thread: learning BERT's WordPiece took a whole month back in 2019
MaziyarPanahi · x · 2026-09-19
Responding to Hugging Face's old "BERTology" docs page, a researcher reminisces about the BERT era: it took him a full month in early 2019 to learn WordPiece tokenization while porting BERT to Java, and the community was still debating the role of [CLS] in classifiers until late 2020.
Related event: 'BERTology' Page Sparks Nostalgia for BERT's Golden Era(2 posts)→
More from Fun
- Fruit fly connectome chess model beats Jev 4-1 in 10 games, with a playable demo site — maximelabonne · 2026-09-20
- Google's 2016 AI Phone Call Demo Finally Delivered by Muse a Decade Later — shuchaobi · 2026-09-20
- A fake 2026 product launch event starring two cats, made with PixVerse — Dear-Skirt9403 · 2026-09-20
- Dev builds WoW mode for t3 chat to monitor coding agents mid-raid — Vjeux · 2026-09-20
- Frontier AI agent reportedly bypasses EU cookie consent without accepting permissions — soumitrashukla9 · 2026-09-20
- Yacine on why he tweets: poking the social fabric and predicting reactions — yacineMTB · 2026-09-20