gpu-lexer: Vercel's 27.5KB WebGPU model does language-agnostic syntax highlighting in the browser
shuding · x · 2026-09-08
Vercel engineer Shu Ding released gpu-lexer from Vercel Labs: a 27.5KB model running on WebGPU for browser syntax highlighting, language-agnostic by design—instead of grammars, a tiny model labels each token using local and whole-file context.
- It splits source into words/whitespace/symbols, labels each part, and merges adjacent labels into spans; it works on languages never seen in training.
- On 10 concatenated copies of three.min.js (5.56M chars), it finished in 471.7ms in-browser vs Prism.js at 1.20s and Shiki at 30.4s, without blocking the main thread.
- Caveats: on held-out files 12.57% of token labels differ from Shiki (agreement, not correctness), and it's an experimental learning project not meant to replace existing highlighters.
More from coding & agent
- Hugging Face ships hf CLI updates: deduped cache saves 19GB, faster buckets, retryable GPU jobs — yb2698 · 2026-10-26
- Fern 0.1 debuts: a model built to audit training data and debug agent traces for AI builders — bneyshabur · 2026-10-26
- Fern 0.1 posts 73.4 on SWE-Bench Verified, far behind frontier models on agentic benchmarks — bneyshabur · 2026-10-26
- One prompt, full Blender animation: dev says he never touched Blender once — iamfakhrealam · 2026-10-26
- Claude Code finds root backdoor in Honda Civic head unit, then debloats it — IndraVahan · 2026-10-26
- Claude Code + Monid MCP builds a $12K/year B2B prospecting engine on live data — aryanXmahajan · 2026-10-26