Reverse-Engineering RK3588 NPU: Open Compiler Runs GPT-2 at 36 tok/s
one_does_not_just · reddit · 2026-08-20
The author shares a year-long effort to reverse-engineer the RK3588 NPU. By cracking the register format and building an open-source compiler + runtime, they achieved model execution directly from PyTorch, ONNX, and JAX without the vendor SDK.
Results:
- Supports GPT-2 and SigLIP on RK3588 NPU.
- GPT-2 inference speed reaches 36 tok/s.
- A detailed technical blog and report are provided, covering everything from memory hacking to compiler architecture.
More from Infra
- Rise of S3-native products as a new developer trend — DanielLockyer · 2026-08-20
- Terminal tool cmux reported for high memory usage — DanielLockyer · 2026-08-20
- Loudoun County Gets Rich From Data Centers, Residents Question the Cost — AndyMasley · 2026-08-20
- Developer Hits One Billion Tokens in a Single Week — jh3yy · 2026-08-20
- Local data centers face NIMBY backlash as 'dystopian betrayal' — ai · 2026-08-20
- Dockerlings: Interactive terminal app to practice Docker — tom_doerr · 2026-08-20