Google open-sources ML Drift: one GPU engine for GLES/OpenCL/Metal/WebGPU, cutting Shorts frame latency 40%
lmoroney · x · 2026-10-09
Google's AI Edge team open-sourced ML Drift under Apache 2.0 — the GPU compute engine inside LiteRT, also shipping as a standalone C++ library.
- Unified cross-platform: one codebase covers OpenGL ES, OpenCL, Metal, and WebGPU, bringing the same acceleration to Android, iOS, web, and desktop.
- New capabilities: 5D tensor support (the old TFLite GPU delegate stopped at 4D), tensor virtualization for shared shaders across backends, and stage-aware kernels that switch between prefill and decode for on-device LLMs.
- Measured gains: YouTube Shorts cut average frame latency by up to 40% after migrating segmentation effects; Google Photos saw up to a 2-second speedup vs the legacy delegate; Adobe and Snap each reported 30% faster on-device features.
- Migration advice: the TFLite GPU delegate gets no new features — route new work through LiteRT ML Drift, and benchmark one production model on a real phone before fully switching.
More from Infra
- 27B model on a single RTX 4090: 262K context at ~130 tok/s with NInfer — Distinct-Pie2389 · 2026-10-09
- boat spins up 250 agent sandbox VMs for 10 cents: full Ubuntu boxes at $20/mo — RexDouglass · 2026-10-09
- A cache hit is not free: inside Triton's compilation cache and hidden costs — Mahmoud_Zalt · 2026-10-09
- AI agents could spawn history's largest bureaucracy, where machines create work for machines — brucemacv · 2026-10-09
- Bittensor-based GPU cloud Lium buys back and burns nearly $2.7M of SN51 tokens in six months — markjeffrey · 2026-10-09
- Inference platform doubles providers since launch as competition pushes prices down — AccBalanced · 2026-10-09