MPK Update: Megakernel Compilation for LLMs

JiaZhihao · x · 2026-07-15

Marking MPK's first anniversary, the author shared several updates:

• Plans to present at OSDI 2026 on how to compile an LLM into a single megakernel.
• The runtime has been rebuilt for Blackwell, with faster and leaner megakernel versions coming soon.
• The community has extended the megakernel to more hardware backends, including a port for AMD MI350X.
• The project now has over 50 contributors.

A cited excerpt explains MPK's core concept: fusing LLM computation and communication into a single GPU megakernel to reduce the difficulty of hand-writing them, claiming a 1.2–6.7x reduction in latency.

Original post →

More from Infra

• China’s AI arms race is increasingly defined by chips, data centers, and open models — BenBajarin · 2026-07-22
• Agent search bottlenecks are now about variance, not raw latency — rohanpaul_ai · 2026-07-22
• Gavin Baker argues Nvidia may be one of open source AI’s biggest supporters — GavinSBaker · 2026-07-22
• AI Power Demand Exposes US Energy Gap, Urging Shift from Scarcity to Abundance — bradneuberg · 2026-07-22
• Gavin Baker says Nvidia’s $630B figure would be system revenue, not all Nvidia’s — GavinSBaker · 2026-07-22
• A Firecracker-based platform says it can host 6,000 AI agents on one 256 GB server — maritime_sh · 2026-07-22

Infra channel →