Dev builds local desktop Jarvis on MiniCPM-o4.5 after cloud API version racked up $861 in fees
面壁智能 · wechat · 2026-09-03
Featured in ModelBest's developer spotlight, Lin Yi's team open-sourced AIJarvis, a local multimodal desktop agent powered by MiniCPM-o4.5's full-duplex capabilities.
- From screenshots to continuous presence: an earlier cloud-based version that sent screenshots every few seconds was slow and cost $861.69 in API fees. The new design feeds one desktop frame plus one second of system audio per second, with the model choosing between LISTEN and SPEAK states — all inference runs locally.
- Gaming companion: warns of approaching mobs in Minecraft, shows low-frequency bubble hints in CS, with per-game prompts and interaction pacing; it never controls the game.
- Study mode: understands lecture video and teacher audio, nudges you when distracted, and auto-generates Markdown notes with key frames, formulas, and slides.
- Daily memory: builds a timeline of your day's activities, spots when you're stuck or overworked, and can render a schedule image via an optional image-generation API.
- Privacy: screen/audio processing and core inference stay on-device; raw data isn't stored long-term and sensing can be paused anytime.
- Packaged for end users: a Windows installer bundles the Python backend and compiled C++ runtime, auto-downloading a 6.32GiB GGUF model on first launch; llama.cpp-omni inference with DXGI screen capture, WASAPI audio, CUDA acceleration with automatic layer allocation, CPU fallback supported.
Project: github.com/LYiHub/pub-local-jarvis
More from Apps
- GEO startup Jam ships marketing agents to boost brand citations across ChatGPT, Perplexity and Gemini — jia_seed · 2026-09-03
- Until Labs Scales Cryoprotectant Search to 250,000 Molecules With AI — NirantK · 2026-09-03
- Grok app is now available on Android, Musk announces — dean_rie · 2026-09-03
- Social Bid launches an X sponsorship marketplace where creators keep 80% — alexmacgregor__ · 2026-09-03
- Playgrnd beta launches: 32 free browser tools for making weird, beautiful things — yuwen_lu_ · 2026-09-03
- Google launches Gemini student discount plan in Japan, open through Dec 31 for one year of perks — heiga_zen · 2026-09-03