Vercel's shuding ships GPU Lexer v0.0.5: WebGPU highlighter hits 92.2% accuracy, 400x faster than Shiki

shuding · x · 2026-11-15

Vercel engineer shuding released gpu-lexer v0.0.5, retraining its model to raise weighted accuracy on the top 25 languages from 86.77% to 92.20%.

How it works: the tool splits source code into simple spans (words, whitespace, symbols), then a tiny WebGPU model labels each span using local and whole-file context—no per-language grammar needed, so it can highlight languages it never saw in training.
Performance: highlighting 10 concatenated copies of three.min.js (5.56M chars) takes just 75.3ms vs 30.4s for Shiki—roughly 400x faster—at a fraction of the library size of Prism.js or Highlight.js.
Caveat: the author notes it's an experiment; agreement with Shiki isn't objective correctness, and results may vary on unseen languages or real-world code.

More from coding & agent