Mobile AI Factory: agents auto-build on-device SLMs with 19x less GPU time
pmttyji · reddit · 2026-11-16
The paper "Mobile AI Factory" (arXiv 2611.13509) presents an agentic system that produces a fine-tuned, quantized on-device SLM for a given task and phone, keeping user data local and offline-capable.
How it works:
- An orchestrator LLM decides at each step whether to tune hyperparameters, mine real data, or generate synthetic examples; fixed rules roll back accuracy-hurting changes.
- It searches from small to large models across bases and quantizations, adapting its accuracy goal, guided by real measurements of the target phone across inference engines and backends.
Results:
- Evaluated on 10 tasks and 4 Android phones: on-device models beat two five-shot frontier models on 5 tasks (by up to 0.34) and match them on 3 more.
- On entity extraction, the search came within 0.03 of exhaustively training all 27 variants while using 19x less GPU time.
- Phone measurements matter: without them, Claude Opus 5.5 with full hardware/model specs misses app requirements in 25% of easy/moderate and 60% of hard cases.
Paper, PDF, GitHub and HuggingFace collection are all public.
Related event: Mobile AI Factory: agents auto-build on-device SLMs with 19x less GPU time(2 posts)→
More from Infra
- Stratechery's Fictional Obituary: Imagining Jensen Huang's 2026 Exit From Nvidia — Gregory_C_Allen · 2026-11-16
- vLLM serves DeepSeek V4.1-Flash at 954K tokens/s per GPU, $38.96/GPU-hour on GB300 NVL72 — bookwormengr · 2026-11-16
- Colossus 2's 1.2M B200s could generate ~$400B/year in tokens at DeepSeek Flash rates — bookwormengr · 2026-11-16
- BIS issues rule blocking remote access to US AI chips via third-country data centers — teortaxesTex · 2026-11-16
- Qdrant 1.20.1: 50% fewer cold-storage reads, ~2x faster quantization, agent-ready setup — qdrant_engine · 2026-11-16
- One 16GB GPU runs songs, voice, images, 720p video and local RAG — all offline and open source — llama-of-death · 2026-11-16