MPK Update: Megakernel Compilation for LLMs

JiaZhihao · x · 2026-07-15

Marking MPK's first anniversary, the author shared several updates:

• Plans to present at OSDI 2026 on how to compile an LLM into a single megakernel.
• The runtime has been rebuilt for Blackwell, with faster and leaner megakernel versions coming soon.
• The community has extended the megakernel to more hardware backends, including a port for AMD MI350X.
• The project now has over 50 contributors.

A cited excerpt explains MPK's core concept: fusing LLM computation and communication into a single GPU megakernel to reduce the difficulty of hand-writing them, claiming a 1.2–6.7x reduction in latency.

Original post →

More from Infra

• SF Compute founder: buying compute is 'an absolutely awful experience' right now — IgorCarron · 2026-09-11
• SmolVM open-sources persistent computer infrastructure for agents that outlive chat sessions — aniketmaurya · 2026-09-11
• PyTorch Day Korea 2026 launches first offline conf, CFP closes Sept 13 — PyTorch · 2026-09-11
• Local LLM server dilemma: 4x CMP-170HX (price up 53% in 20 days) vs Mac Studio M5 Ultra — rumboll · 2026-09-11
• llama.cpp lands Flash Attention tuning for RDNA4, big prefill gains on AMD — pmttyji · 2026-09-11
• Your p99 latency benchmark may be lying: a deep dive into coordinated omission — Franc0Fernand0 · 2026-09-11

Infra channel →