Gave my coding agent hands in my real browser — the hard part was verifying the click landed
Free-Plantain4841 · reddit · 2026-08-17
The author open-sourced (MIT, free) a tool letting coding agents drive your real browser, noting that headless clones hit sign-in walls while real-browser setups have a subtle failure: the agent reports "clicked" but nothing happened — usually a cookie banner or modal backdrop swallowed the click, yet the tool returned success.
The fix is hit-testing before clicking: descend through open and closed shadow roots to find what is genuinely painted at that pixel; if covered, scroll clear of pinned bars or give the covering layer pointer-events: none for one click and restore styles, keeping the same real CDP click so isTrusted stays true.
Hover and drag are first-class operations to reach hover menus, sliders and canvas apps; it halts before consequential actions (posts, payments, passwords, 2FA) and hands control back — an agent holding your session cookies shouldn't publish in your name. Works with Claude Code, Gemini CLI and Codex. Known caveat: overlay piercing classifies layers by geometry/style, not semantics.
More from coding & agent
- Writing C code manually with AI autocomplete beats AI generation — tetsuoai · 2026-08-17
- LlamaIndex releases LlamaExtract agent with 94% accuracy on long docs — llama_index · 2026-08-17
- Dizko open-sources city-events MCP server with 18 tools across 32 cities — eye_tawnyah · 2026-08-17
- Dev Rant: Juggling Multiple AI Chats is More Tiring Than Coding — yihui_indie · 2026-08-17
- Open-source zxLLM predicts LLM VRAM usage & KV-cache needs with high precision — Capable_Item_5918 · 2026-08-17
- Hermes Agent fixes desktop media stream credential leak — Teknium · 2026-08-17