StreamEmbed: on-device real-time screen history indexing, 3.7× faster than screenshot encoding

shangbinfeng · x · 2026-11-20

StreamEmbed is a fully on-device system for real-time visual indexing of your screen history — query your laptop to find that paper with pink-and-blue Pareto curves you saw last week. OCR indexes are fast but miss pixel-only information; full-screenshot encoding is accurate but takes 2.3s per screenshot on a laptop GPU. StreamEmbed gets both: up to 3.7× faster than full-screenshot encoding at matched accuracy, and +6.3 Recall@10 over OCR. Key insight: screens barely change, so temporal + spatial token pruning skips redundant blocks. Benchmarks (SME-H1/H2/Agent) show StreamEmbed dominating the latency-recall frontier vs SigLIP2 and Qwen3 baselines. Installable now to query your own computer-use history.

Related event: StreamEmbed: Open-Source On-Device Visual Indexing of Screen History, 3.7x Faster(4 posts)→

Original post →

More from coding & agent

coding & agent channel →