StreamEmbed: on-device real-time screen history indexing, 3.7× faster than screenshot encoding
shangbinfeng · x · 2026-11-20
StreamEmbed is a fully on-device system for real-time visual indexing of your screen history — query your laptop to find that paper with pink-and-blue Pareto curves you saw last week. OCR indexes are fast but miss pixel-only information; full-screenshot encoding is accurate but takes 2.3s per screenshot on a laptop GPU. StreamEmbed gets both: up to 3.7× faster than full-screenshot encoding at matched accuracy, and +6.3 Recall@10 over OCR. Key insight: screens barely change, so temporal + spatial token pruning skips redundant blocks. Benchmarks (SME-H1/H2/Agent) show StreamEmbed dominating the latency-recall frontier vs SigLIP2 and Qwen3 baselines. Installable now to query your own computer-use history.
More from coding & agent
- Drex Router cuts small-game build cost to $1.26 vs $13.44 on Claude Opus alone — testingcatalog · 2026-11-20
- User teaches AI companion to ignore "drop it" said to his dog, then fixes the over-pause — manosaie · 2026-11-20
- NVIDIA Dynamo cuts RL rollout bottlenecks with KV-aware routing and delta weight updates — NVIDIAAI · 2026-11-20
- LangChain Academy Answers: Can a Better Model Fix a Broken Agent? — LangChain · 2026-11-20
- Agents pull AI compute back to the CPU: AWS, Google, Microsoft, NVIDIA all build on Arm — kimmonismus · 2026-11-20
- Coding agent index shows max effort costs 6x for fewer points; Gemini 4 Neon wins on value — Arindam_1729 · 2026-11-20