EdgeKernelBench: LLM-Generated Kernels Hit 64.5% Correct but Only 25.5% Fast on Edge Backends
encodelab · hf · 2026-10-21
Researchers from Westlake University and Zhejiang University introduce EdgeKernelBench and LEKA (Layered Edge Kernel Agent), the first systematic study of LLM kernel generation across heterogeneous edge backends.
Key points:
- Edge deployment differs sharply from server GPUs: heterogeneous hardware, fragmented software stacks, tight resource budgets;
- The benchmark uses a unified IR with a compiler interface to each backend, instantiated with TVM as reference compiler, spanning operator-level to application-level scopes;
- Core finding: with a single generation attempt, the best model reaches 64.5% Correct@1 but only 25.5% [email protected] — functional correctness does not translate into performance gains;
- LEKA organizes optimization through a layered IR, achieving 43.5% [email protected] vs 13.0% for direct generation and 26.1% for Claude Code on the same base model and budget.
Project page: pillar9.github.io/EdgeKernelBench
More from Infra
- xAI said to lean on ACC/AEC and copper as Chinese pluggables stay off-limits — jwt0625 · 2026-10-21
- Packed a16z SF infra event features fal, ElevenLabs, BFL and Krea engineers — venturetwins · 2026-10-21
- Baseten ships DeepSeek v4.1 Flash, Reflection Beam 1.0, Nemotron Super 3.5 VL day-zero with claimed top benchmarks — baseten · 2026-10-21
- Modal Engineers Detail Sticky Serverless Routing for Real-Time AI Inference — blelbach · 2026-10-21
- Meta Open-Sources Dist-MoE, Speeding MoE Training Up to 1.40x on GB300 NVL72 — ezyang · 2026-10-21
- PyTorch accelerator integration group details progress and priorities at PyTorchCon — PyTorch · 2026-10-21