Vercel open-sources gpu-lexer: a 27.5KB WebGPU model for syntax highlighting
Vercel engineer Shu Ding has open-sourced gpu-lexer under Vercel Labs, an experimental project: a tiny 27.5KB model running on WebGPU that performs syntax highlighting in the browser, positioned as a learning-oriented experiment.
Confirmed
- The model is language-agnostic and doesn't rely on grammar rules; it guesses code syntax from local and whole-file context. The model itself accounts for about 70% of the library's size and is inlined directly in the bundle.
- Training covers 50+ languages, and the author says it performs reasonably on languages outside the training set. Benchmarked against Shiki (the industry gold standard based on real ASTs), comparing weighted accuracy across the 25 most popular languages on GitHub.
- Performance advantages come mainly from WebGPU: gains grow with source code size, and it doesn't block the main CPU thread.
- The author says the motivation came from years of pain points: huge language grammar bundles, complex language detection and lazy loading, and poor performance from existing solutions slowing down host apps.
Not yet confirmed
- Specific accuracy figures and per-language details aren't given in the material; the actual weighted accuracy numbers need to be checked in the original project write-up.
Why it matters
- gpu-lexer shows the possibility of "small model + WebGPU" replacing traditional regex/AST highlighting pipelines, balancing size and performance—a useful reference for in-browser code display scenarios like documentation sites and online editors. The author cautions, however, that each highlighter's pipeline and use case differ, and this experimental project cannot yet fully replace existing solutions such as Shiki.
2026-09-08 ~ 2026-09-09 · 7 related posts
Primary sources
- [source] gpu-lexer: Vercel's 27.5KB WebGPU model does language-agnostic syntax highlighting in the browser — shuding · 2026-09-08
- gpu-lexer is fast via WebGPU and off-main-thread, but not yet a replacement, author says — shuding · 2026-09-08
- [source] gpu-lexer author: trained on 50+ languages, model is ~70% of the 27.5KB bundle — shuding · 2026-09-08
- [source] ML syntax highlighter matches Shiki accuracy with the model inlined in the bundle — shuding · 2026-09-08
- Vercel engineer builds a new syntax highlighter, benchmarks correctness against Shiki — shuding · 2026-09-08