Fully offline camera app runs YOLOv8n + fine-tuned 0.8B VLM on-device, open-sourced

divinetribe1 · reddit · 2026-09-13

A developer open-sourced (GPL 3) a fully offline camera app whose architecture centers on a two-model split:

Detection: YOLOv8n trained on Open Images V7, 601 classes, live on preview frames; tflite fp16 on Android
Description: a fine-tuned 0.8B VLM quantized to 4-bit via MLX Swift on iOS and llama.cpp + mtmd on Android, answering in one sentence (e.g. "this is a bill from Umpqua Bank for $76.50, due April 2nd, 2027")
Routing: before the VLM runs, a document check (Vision document segmentation on iOS, ML Kit text recognition on Android) decides page-question vs scene-question — same weights, different prompt, noticeably better answers
Numbers: 300ms model load; 90s per description on a cheap GPU-less Android phone; Vulkan probe-and-fallback; arm64 only
All weights ship with the app — no first-run download, no API keys

More from coding & agent