Claude-written megakernels for 8 open models hit up to 1.77x speedup over vLLM/SGLang

joao_gante · x · 2026-11-19

A team had Claude write a separate megakernel for each of 8 open models (8B to 594B): one GPU kernel that runs the entire decode loop, declaring the end of one-size-fits-all inference engines.

The approach replaces generic inference engines with per-model full-loop kernels — an aggressive direction in inference-stack optimization.

Original post →

More from Infra

Infra channel →