BiRefNet-lite runs fully on a phone NPU at 1024px in 23.7ms on Snapdragon 8 Elite Gen 5, open weights
subhashg547 · reddit · 2026-11-15
An independent researcher rewrote BiRefNet-lite to run tap-to-lift subject cutout entirely on a phone NPU. The original wouldn't compile: 20 deformable convs, 5D/6D Swin tensors, cyclic-shifted windows, and 1024px activations exceeding on-chip memory.
Approach: exact graph rewrites (identical output) — 4D window partition, roll-free shifted windows, per-head attention; architecture changes — plain convs instead of deformable, 640px backbone with 1024px mask from two inputs, a slim full-res tail, one global window in Swin stage 4; fully 8-bit QAT distillation through the exported NPU graph.
Results: 23.7ms for 1024×1024 masks on a Galaxy S26 (Snapdragon 8 Elite Gen 5), 31.1ms on S25, 182ms on a mid-range 7 Gen 1 — all 2,060 layers on NPU. Stock BiRefNet-lite OOMs at 1024px and falls back to CPU at 512px (677ms, 2.1GB). The int8 model matches its fp32 teacher on held-out ECSSD/DUT-OMRON S-measure.
Gotcha: during distillation a sigmoid gate on the deepest decoder branch saturated to 1e-11 and the branch died silently — the mask loss showed nothing because the rest compensated. Restored from teacher with a feature loss.
Weights, compiled files for 42 Snapdragon chips, and code are MIT-licensed on GitHub/HF. The author is also seeking a CS endorsement for arXiv.
More from Infra
- ggml tensor library trends on GitHub with 15.8k stars, up 154 in a day — ggml-org · 2026-11-15
- speech.cpp v0.11.0 lands Haru-4B, a full-duplex voice chat model running locally in C++ — alexcovo_eth · 2026-11-15
- Open-source h3metal runs image gen on M5/M6 matrix units, 2.1-2.3x faster than Draw Things — audenvenn · 2026-11-15
- Azure outage draws silence: big consultancies avoid vendor concentration risk talk — DavidLinthicum · 2026-11-15
- 2 Starship launches equal 20 Falcon 9 missions as 52 V3 sats match 500+ V2 capacity — XFreeze · 2026-11-15
- HPE Beats Estimates but Stock Slides as Analysts Doubt AI Server Margins — DavidLinthicum · 2026-11-15