Visual Token Routing cuts computer-use agent inference cost up to 3.3x with 2.5-5.8x fewer tokens
wellecks · x · 2026-11-10
A new arXiv paper by Pranjal Aggarwal and Sean Welleck introduces Visual Token Routing (VTR), attacking the key cost bottleneck of computer-use agents: each screenshot becomes thousands of visual tokens that pile up over long interaction histories.
- Insight: most visual tokens are semantically irrelevant to the next action or temporally redundant across screenshots.
- Method: jointly train a VLM with a lightweight spatio-temporal router that decides, per image region across current and past screenshots, whether its token enters the language model, optimized with an adaptive loss to minimize tokens with minimal degradation.
- Results: with just 2,000 training steps, VTR matches the full-resolution model on OSWorld, CUA-World and video-game action prediction while using 2.5-5.8x fewer visual tokens — cutting inference cost up to 3.3x and speeding prefill up to 2.9x, beating resolution-lowering baselines.
More from coding & agent
- WorkflowArena: First Benchmark for Multi-Agent Workflow Orchestration With 145 Real-World Tasks — _akhaliq · 2026-11-10
- Klint 1.0 ships: AI-ported TS to Rust is 25x faster with 28x less memory — rchardkovacs · 2026-11-10
- Pi's Virtual Models auto-route queries between local Qwen and remote Kimi or GLM — ggerganov · 2026-11-10
- Unlimited-OCR-2 tested: parsing 30-page PDFs in one request without breaking tables — KhuyenTran16 · 2026-11-10
- Screaming Frog tip: your AI assistant can fully configure your SEO crawler — VeryWellVersed · 2026-11-10
- Managing agents like the infamous "plz deal" VP: dump context, give minimal direction — curious_vii · 2026-11-10