Sightline: MIT-licensed JS package gives in-app agents page context without screenshots
Wei_Sushi · reddit · 2026-10-28
Developer WeiSushi released Sightline, an MIT-licensed JS package for assistants embedded in web apps.
- Local DOM parsing: reads element labels, roles and control states (empty fields, disabled buttons, dialog names) and returns a compact text outline capped at 6,000 characters by default.
- Replaces screenshot-description calls: extraction involves no model calls, so it can eliminate an entire screenshot→description pipeline; it still can't replace vision for charts, images or layout issues.
- Watches page changes and highlights controls by reference, helping agents walk users through forms.
- Known limits: typed field values are excluded, other page text still needs privacy handling, and explaining why a button is disabled still requires the product's rules.
Repo and bookmarklet demo in the comments; the author is soliciting feedback on where it breaks down in real apps.
More from coding & agent
- LLMs supply intelligence, agents do the work — do SaaS interfaces become obsolete? — ThomasBuildLab · 2026-10-28
- Codex scans your connectors/MCPs unprompted in the background, and you can't tell — kenziyuliu · 2026-10-28
- UCLA and Tsinghua paper probes how far coding agents can go in visual generation — ucla-cmllab · 2026-10-28
- 'Claude Duet has won': devs say use provider-agnostic harnesses as the top model keeps rotating — p_e_cooper · 2026-10-28
- ArkenOS deploys swarm of autonomous scientists; Sam Altman approves — _AustinCalvert_ · 2026-10-28
- Lucas Meijer demos cloud agent workspace running parallel Opus tasks at AI Tinkers meetup — lucasmeijer · 2026-10-28