StreamEmbed: On-device real-time visual indexing of your screen history, 3.7x faster
CShorten30 · x · 2026-11-19
Researchers released StreamEmbed, an open-source system that indexes your screen history in real time entirely on-device, so you can ask your agent to find anything you've glimpsed—an old paper screenshot, a video you watched at lunch.
- Key idea: screens barely change frame to frame, so spatial token pruning skips recomputing blocks that repeat, avoiding full-screenshot re-encoding
- Performance: up to 3.7x faster than full-screenshot encoding at matched accuracy, and +6.3 Recall@10 over an OCR-only index (OCR is fast but misses pixel-only information; screenshot indexing is accurate but costs 2.3s per frame on a laptop GPU)
- Ships as an Apache-2.0 repo with a paper and an MCP server; multimodal, multilingual, private
Led by @dayoon12161 with collaborators from the on-device personal agents space.
Related event: StreamEmbed enables real-time visual indexing of screen history(2 posts)→
More from coding & agent
- GitHub adds Adaptive mode to Copilot code review that picks review effort per PR — mariorod1 · 2026-11-19
- Gemini 4 Neon takes #2 on LMArena agent index at a fraction of Claude's cost — lyraxana · 2026-11-19
- Codex's tightened sandbox can no longer run bind tests, developer complains to OpenAI — kristoph · 2026-11-19
- She built a full custom membership site just by talking to Claude — evielync · 2026-11-19
- New TLAPS-Bench blog: formally proving correctness of real-world system specs with TLA+ — tianyin_xu · 2026-11-19
- Pierre releases Words, a streaming Markdown parser 2x faster than md4x and 20x faster than Streamdown — andersonbcdefg · 2026-11-19