Google says edge AI is RAM-bound, not compute-bound, and tiny models fit on devices

AI Engineer · youtube · 2026-07-26

Google AI Edge argues that the real constraint on edge AI is RAM, not compute, and says that problem is getting worse as device memory gets tighter.

• A 2B Gemma model quantized to 2.9 bits per weight runs on a Raspberry Pi at about 8 tokens/sec and on Qualcomm NPUs fast enough for a few video frames per second.
• Even smaller 500M-to-50M models are aimed at older laptops and cheap devices, where fine-tuning matters more than prompting.
• One production example is an offline voice dictation app built on two sub-billion Gemma models that also remove filler words.
• The talk also shows an AI Edge Gallery demo and a hobby robot example.

Original post →

More from Embodied

• Robot goes to the fridge and fetches a beer — Darpinian · 2026-07-27
• Researchers show digital circuits can be replicated with knitted fabric — mtizard · 2026-07-27
• Local Qwen models power a robot that tests 78 smartphones’ battery life — gappyvalley · 2026-07-27
• TechCrunch says brain-wave signals could be the next unlock for physical AI training — TechCrunch AI · 2026-07-27
• YC talk says robots need better pretraining, memory, and compositional behaviors — ycombinator · 2026-07-27
• China’s drone-license boom cools as training centers shrink and professional demand takes over — 创业邦 · 2026-07-27

Embodied channel →