Gelato open-sources an Apache-2.0 protocol linking AI models to phones, glasses, and cameras
Milo_Bindu · reddit · 2026-11-08
StreamingTelligence open-sourced Gelato, an Apache-2.0 protocol and SDK stack for connecting AI systems to phones, smart glasses, cameras, and embedded devices, addressing the pain of needing a separate integration for every model–device combination.
The architecture splits into three parts:
- A device agent advertises the device's capabilities
- A controller handles pairing, identity, routing, media, commands, cancellation, and policy
- An optional model runtime interprets device input or proposes actions
A key security boundary: the model never receives device-pairing credentials and cannot execute actions directly—it only proposes, while the controller and device retain routing and execution control.
The MVP supports Android, iPhone, Rokid smart glasses, and the Seeed XIAO ESP32S3 Sense camera, with integrations for Qwen Live, Codex, provider-neutral HTTP, and StreamingTelligence. Repo on GitHub, with a 1:46 demo video.
More from coding & agent
- Dev Uses AI Agent to Branch, Update and Wire Two PowerShell Modules in One Pass — dfinke · 2026-11-08
- mitsuhiko: djs' JS "heap dumps" may beat real dumps for durable coding agents — mitsuhiko · 2026-11-08
- Claude Code builds 1M-zombie GPU crowd simulation in one session, 12s clip in 21 min — ravisparikh · 2026-11-08
- 2026 AI office agent landscape: 150 companies, 16 layers, model labs selling straight to enterprises — FinanceYF5 · 2026-11-08
- exedev quietly launches per-minute sandbox VMs via 'ssh exedev new --sandbox' — davidcrawshaw · 2026-11-08
- Vibe-designed programming languages flood in, and this PL veteran is not happy — zetalyrae · 2026-11-08