Vercel open-sources gpu-lexer: a 27.5KB WebGPU model for syntax highlighting in the browser
Vercel engineer Shu Ding open-sourced gpu-lexer under Vercel Labs: an experimental, language-agnostic model of only 27.5KB that runs on WebGPU and performs syntax highlighting directly in the browser. The project drew attention and reshares from developers including shadcn and Aiden Bai.
Confirmed
- The model is language-agnostic: instead of relying on grammar rules, it guesses syntax from local and whole-file context. The model itself takes up about 70% of the library size and is inlined directly in the bundle.
- It was trained on 50+ languages, and the author says it performs decently on languages outside the training set. It was benchmarked against Shiki (the AST-based, near-gold-standard highlighter) using weighted accuracy over the top 25 languages on GitHub.
- The performance edge mainly comes from WebGPU: the larger the source file, the bigger the advantage, and it does not block the main CPU thread.
- Shu Ding cited long-standing pain points as motivation: huge grammar bundles, complex language detection and lazy loading, and poor performance slowing down host apps.
Unconfirmed
- Concrete accuracy numbers and per-language details were not given in the materials. DoctorGester's post indicates the benchmark methodology—using Shiki as the gold standard with weighted accuracy over popular languages—drew community skepticism.
Why it matters
- gpu-lexer demonstrates the feasibility of replacing traditional regex/AST highlighting pipelines with a small model plus WebGPU, balancing bundle size and performance for browser-side code display (docs sites, online editors). The author himself cautions that highlighter pipelines and use cases differ, so this experiment cannot yet fully replace Shiki and other existing solutions.
2026-09-08 ~ 2026-09-09 · 9 related posts
Primary sources
- [source] gpu-lexer: Vercel's 27.5KB WebGPU model does language-agnostic syntax highlighting in the browser — shuding · 2026-09-08
- gpu-lexer is fast via WebGPU and off-main-thread, but not yet a replacement, author says — shuding · 2026-09-08
- [source] gpu-lexer author: trained on 50+ languages, model is ~70% of the 27.5KB bundle — shuding · 2026-09-08
- ML syntax highlighter matches Shiki accuracy with the model inlined in the bundle — shuding · 2026-09-08
- [source] Vercel engineer builds a new syntax highlighter, benchmarks correctness against Shiki — shuding · 2026-09-08
- Syntax Highlighting Benchmark Using Shiki as Ground Truth Draws Accuracy Dispute — DoctorGester · 2026-09-09