EdgeKernelBench: Unified LLM-Driven Kernel Generation for Edge Backends
Yuhua Zheng, Xueyi Chen, Xinze Zou, Xin Li, Haoji Hu, Huan Wang
cs.LG
2026-10-17
EdgeKernelBench: on phone CPU, Jetson, and RK3588, GPT-5.6-sol one-shot scores 64.5% Correct@1 but 25.5% [email protected] vs tuned TVM. LEKA raises that to 43.5% on 69 tasks, from 13.0%.
LLM kernel generation already has a fixed test. Take a PyTorch reference, emit CUDA or Triton, and race a compiler. The setup assumes a tidy software stack and enough memory, and the objective is speed. Edge boards do not match that. A Snapdragon phone, a Jetson, and a Rockchip board each bring their own processor and runtime, and they share RAM with the rest of the system. One oversized intermediate can make a model impossible to deploy.
KernelBench stays on server GPUs. MobileKernelBench reaches phones, but its public tasks are mostly single operators and attribute variants. They do not test fusion across operators, or memory planning on a small model graph. A new backend means a new language and a new set of legality rules.
EdgeKernelBench pairs one shared IR with tasks that grow in scope. There are 331 canonical tasks, all static PyTorch programs checked by hand. Level 1 has 128 basic operators, Level 2 has 162 fused modules, and Level 3 has 41 compact model graphs, including MobileNet, ResNet-18, NanoDet, and decoder blocks from Llama and Qwen2.
The reported runs do not cross every domain with every device. A Snapdragon 8 Gen 3 phone CPU on LLVM runs the mobile set, 204 tasks. A Jetson AGX Orin 64GB on CUDA runs the robotics set, 218 tasks. A Mali-G610 on the RK3588, via OpenCL, runs the AIoT set, 228 tasks. That is 650 instances. Device and task mix move together.
The model fills in a unified kernel IR with computation, scheduling, and memory choices. TVM 0.23.0 lowers it, legalizes it, and runs it. The compiler does not search. One-shot generation is a single call. Correctness checks structure, shape, dtype, and numeric tolerance on three random seeds. Timing alternates the candidate and the baseline in one session. Compilation and deployment prep stay off the clock. [email protected] is the share of tasks that are correct, have valid timings, and whose median paired speedup is above 1. The baseline is TVM after MetaSchedule search on the device. Calibration caps representative programs at 1,000 on-device trials on CPU and 1,500 on GPU; task budgets are then set from how soon search converged.
A single call has to hold both the graph and the thread map, and a slow timing does not say which decision to edit. LEKA splits agents along the IR. L0 edits graph structure, layout, and fusion in Relax. L1 edits loops and parallelism in TIR. L2 edits vectorization and registers in CUDA C, OpenCL C, or LLVM IR. Lowering between layers keeps semantics and does not search. A read-only hardware agent profiles the device and sends the bottleneck back to the layer that owns it. A candidate tree keeps older versions. The Claude Code baseline uses the same GLM-5.3-Flash and the same wall clock: 90 minutes on Level 1, 155 minutes on Levels 2 and 3. The subset is stratified, 23 tasks per backend, 69 in total.
Weight Table 1 by task count and the abstract totals line up with GPT-5.6-sol: one shot, all 650 tasks.
| Comparison | Metric | Result |
| GPT-5.6-sol, 650 tasks, vs tuned TVM | Compile@1 / Correct@1 / [email protected] | 73.5% / 64.5% / 25.5% |
| Best one-shot [email protected] across three backends | Level 1 / 2 / 3 | 51.0% / 16.2% / 4.0% |
| Best Level 1 Correct@1 | CPU / CUDA / OpenCL | 94.5% / 79.6% / 73.5% |
| DeepSeek-V4-Pro, CPU Level 2 | Correct@1 / [email protected] | 75.3% / 3.2%; GPT's [email protected] on that slice is 15.1% |
| MNN 3.3.0 (176 CPU tasks, 203 OpenCL) | Best [email protected] | L1: 50.7% CPU, 31.6% GPU; L2 14.9%; L3 5.3% |
| TensorRT 10.3.0 (215 of 218 CUDA tasks) | Best [email protected] | L1 43.2%, L2 5.3%, L3 none; at 2× the best share is 7.4% |
| 69 tasks, GLM-5.3-Flash for every agent | [email protected] vs tuned TVM | One-shot 13.0%, Claude Code 26.1%, LEKA 43.5% |
| Same 69 tasks | Correct@1 / Compile@1 | 42.0% to 100%, 60.9% to 100% |
| LEKA vs Claude Code, geometric mean | L1 / L2 / L3 / overall | 3.4× / 4.7× / 2.9× / about 3.8× |
The geometric mean keeps programs that got slower. On Level 3 the best [email protected] on every device sits between 0 and 4%. CPU correctness is higher than GPU correctness because thread partitioning and parallel mapping are harder to get right. CUDA [email protected] beats OpenCL, and at Level 1 every model shows that pattern. Gemini-3.7-Flash has the highest Level 3 correctness on all three backends. On CPU Level 2, DeepSeek-V4-Pro beats GPT on correctness and still loses on speed.
Programs that beat MNN or TensorRT cluster on single operators. No Level 3 candidate beats TensorRT. Pooled across models and backends, memory errors are 34.0% of failures, syntax and API errors 25.4%, numeric mismatches 18.9%, and parallel errors 12.0%. Most memory failures are illegal buffer scopes, not a device that ran out of room.
On the 69 tasks, LEKA's [email protected] is 34.8% on CPU, 30.4% on CUDA, and 65.2% on OpenCL. Claude Code scores 21.7%, 26.1%, and 30.4%. CUDA barely moves. OpenCL more than doubles. Geometric-mean speedup versus Claude Code is 8.60× on CPU, 2.46× on CUDA, and 2.49× on OpenCL. Pooled, Level 2 (4.7×) beats Level 1 (3.4×), and Level 3 falls back to 2.9×.
SqueezeNet 1.1 on the RK3588 shows the layers stacking. Relax switches to NHWC and rewrites 1×1 convolutions as matrix multiplies. TIR tiles through shared memory and moves speedup over the initial candidate from 1.3× to 1.7×. OpenCL register-blocks those matmuls and reaches 3.5×. The denominator is LEKA's own starting point, not TVM. Large speedups against a strong baseline show up on a few one-shot tasks. Gemini takes an OpenCL ConvNeXt block from 18.0 ms to 0.6 ms, 30.9× over TVM, by splitting depthwise convolution from the channel path. GPT fuses the whole block and lands at 0.9×. A CPU dequantized matmul goes from 39.7 ms on MNN to 2.1 ms, 19.0×, and skips a 44 MiB floating-point weight tensor.
Someone adding an edge operator needs two answers. Can the optimization move to another board, and how fast does quality fall once the program grows past one kernel? A shared IR separates reusable decisions from backend syntax, so a new board is mostly a new adapter. LEKA and Claude Code share the model and the clock. [email protected] moves from 26.1% to 43.5% because the search is split by IR layer.
The step is incremental. One-shot generation beats tuned TVM on 25.5% of 650 tasks. After LEKA makes all 69 programs correct, fewer than half are faster than TVM.
The appendix names three follow-ups: compiler integration, coordination between layers and agents, and NPU or heterogeneous execution. The experiments cover LLVM, CUDA, and OpenCL only.
Four Level 3 tasks per backend is a thin base for a 2.9× mean. LEKA runs on GLM-5.3-Flash, not on GPT-5.6-sol, so 43.5% on 69 tasks is a different exam from 25.5% on 650. Each backend is tied to one application domain. The CPU correctness edge mixes the task mix with the fact that a CPU has no GPU thread map.
The introduction treats memory and latency as joint goals. The main tables report compile rate, correctness, and speedup. There is no systematic peak-memory number. The 44 MiB case is one task. A single task can spend 155 minutes of wall clock plus repeated on-device runs, and that cost is not compared with MetaSchedule's thousand-scale hardware trials. [email protected] only asks whether speedup exceeds 1. Two fully traced LEKA programs beat tuned TVM by 1.2× on CUDA (16.2 µs versus 19.8 µs) and 1.6× on CPU (17.0 ms versus 27.5 ms). A win can be thin.