Visualizing tokens and IDs from Borges' Library of Babel with a tokenizer tool

menhguin · x · 2026-09-26

poetengineer built "the tokenizer," a small visualization that extracts tokens and their IDs from a passage of Borges' The Library of Babel. A responder muses about wanting the AI bubble to pop so they can spend more time making things like this with models like Opus.

Original post →

More from Fun

Fun channel →