Dev cuts local AI inference power from 100W to 25W with only a 1.9s→2.8s slowdown
AryanEmbered · reddit · 2026-10-28
A developer built a local-first discipline tracker around a 2B multimodal model: the camera periodically snaps a photo, the local model describes the scene and rates compliance with your schedule, then compiles reports like "you left your desk x times during study hour." A privacy-sensitive workload that genuinely suits on-device inference.
The problem and fix
- By default the GPU burned a constant 100W with fans ramping — unusable on laptops
- Via AMD Adrenaline (power limit down, zero-RPM off, fans locked low), total draw dropped from 100W to 25W
- Cost was modest: inference (current + 3 past frames) went from 1.9s to 2.8s average
Open industry question
- Regular users can't tune any of this; despite NPUs shipping in consumer SoCs, there's no OS-level API to signal a sustained constant-inference workload for power/clock scheduling
- He also rebuts common suggestions: intermittent inference still spikes clocks, llama.cpp doesn't support NPUs, and NPUs draw a lot of power under sustained load
More from Infra
- SemiAnalysis: OpenAI reportedly negotiating ~1GW TPU commitment while seeking sites — dylan522p · 2026-10-28
- I tested 11 video upscaling methods for AI video: local, Topaz and cloud compared — lopas99 · 2026-10-28
- Thermo-backed CTO says AI is in its dial-up phase, touts disaggregated compute for an ultrafast 'broadband moment' — beffjezos · 2026-10-28
- DeepSeek-V4.1-Flash Q2 runs at 41 tok/s on one RTX 3090, 90% token match vs API — musicurgy · 2026-10-28
- USearch v3 RC ships after 1,000 days, its libraries now embedded in 550K-star projects — viglovikov · 2026-10-28
- Will local-model progress outrun GPU depreciation? Reddit debates the buy-now window — Sensualities · 2026-10-28