Google says edge AI is RAM-bound, not compute-bound, and tiny models fit on devices

AI Engineer · youtube · 2026-07-26

Google AI Edge argues that the real constraint on edge AI is RAM, not compute, and says that problem is getting worse as device memory gets tighter.

• A 2B Gemma model quantized to 2.9 bits per weight runs on a Raspberry Pi at about 8 tokens/sec and on Qualcomm NPUs fast enough for a few video frames per second.
• Even smaller 500M-to-50M models are aimed at older laptops and cheap devices, where fine-tuning matters more than prompting.
• One production example is an offline voice dictation app built on two sub-billion Gemma models that also remove filler words.
• The talk also shows an AI Edge Gallery demo and a hobby robot example.

Original post →

More from Embodied

• ROS names Alejandro Hernández Cordero new project lead, eyes 2027 release — ___Mufasaa · 2026-10-27
• ROS Meetup coming to Google's NYC office Oct 29 with speakers from Tangent, Cohesive Robotics, Intrinsic — kscottz · 2026-10-27
• Tesla officially expands robotaxi service area across much of Houston, Texas — Polymarket · 2026-10-27
• Walden Robotics teaches robots to call for help, blending remote assist with autonomy — adnothing · 2026-10-27
• Folding-screen hinges: dozens of ultra-precise parts, one grain of sand can kill the panel — BenBajarin · 2026-10-27
• No π0 weights changed: Motoniq layer lifts real-robot precision fitting from 0/10 to 9/10 — hbouammar · 2026-10-27

Embodied channel →