Google open-sources ML Drift, a cross-platform on-device GPU inference engine
pmttyji · reddit · 2026-10-09
Google's AI Edge team open-sourced ML Drift, a high-performance cross-platform GPU compute engine for on-device AI/ML inference, under Apache 2.0.
Key points:
- Abstracts hardware and low-level APIs across OpenGL ES, OpenCL, Metal, and WebGPU
- Targets real-time interactive ML experiences, from advanced video effects to generative AI
- Serves as the core GPU acceleration engine inside LiteRT, and is also available standalone for custom graphics/inference runtimes
Code is live at github.com/google-ai-edge/ml-drift.
Related event: Google Open-Sources On-Device GPU Inference Engine ML Drift(2 posts)→
More from Infra
- DuckDB v2.0 CLI agent mode cuts agent-read tokens by 59% on TPC-H benchmarks — josh_wills · 2026-10-10
- Datology releases Zephon, a deterministic on-the-fly dataloader born from MosaicML Streaming's legacy — josh_wills · 2026-10-10
- Tsinghua's TokenRouter: Token-Level LLM Routing Hits Up to 64.15X Serving Throughput — rohanpaul_ai · 2026-10-10
- Meta Muse Auto-Routes to OpenRouter Free Models for Zero-Cost Long Tasks — sven_ai · 2026-10-10
- DIY hybrid GPU/CPU/SSD rig cuts DeepSeek TTFT from 75s to 8.9s at 16K prefill — HankYeomans · 2026-10-10
- vLLM thread (4/5): locality-domain MoE sharding speeds up decode 1.2x — vllm_project · 2026-10-10