gpu-lexer: Vercel's 27.5KB WebGPU model does language-agnostic syntax highlighting in the browser
shuding · x · 2026-09-08
Vercel engineer Shu Ding released gpu-lexer from Vercel Labs: a 27.5KB model running on WebGPU for browser syntax highlighting, language-agnostic by design—instead of grammars, a tiny model labels each token using local and whole-file context.
- It splits source into words/whitespace/symbols, labels each part, and merges adjacent labels into spans; it works on languages never seen in training.
- On 10 concatenated copies of three.min.js (5.56M chars), it finished in 471.7ms in-browser vs Prism.js at 1.20s and Shiki at 30.4s, without blocking the main thread.
- Caveats: on held-out files 12.57% of token labels differ from Shiki (agreement, not correctness), and it's an experimental learning project not meant to replace existing highlighters.
More from coding & agent
- Code is no longer the product: Milanović on software engineering when agents write the code — rseroter · 2026-09-09
- GPT-6 Astra codes geometry, Blender + Clay plugin renders, Dreamina finalizes footage — thetripathi58 · 2026-09-09
- Devs hack internal Claude Code plugin repos to share skills with teams — gregbarbosa · 2026-09-09
- Claude skips the training set: HTML/CSS rendered in a headless browser makes his site images — shashib · 2026-09-09
- TrueForge: open-source agent harness turns an LLM into a working agent, 5.3k stars — dr_cintas · 2026-09-09
- State Machines launches stateful API replicas for testing enterprise AI agents at scale — SimplyAnnisa · 2026-09-09