SwitchMem dynamically picks text vs visual memory per step for GUI agents
KoreaUniversity · hf · 2026-10-14
Researchers from Korea University and collaborators studied memory modality selection in VLM-based GUI agents and proposed SwitchMem.
- Finding: existing memory-augmented GUI agents use fixed representation strategies, but systematic experiments across GUI benchmarks show the preferred memory representation is strongly step-dependent — no single modality is consistently optimal.
- Method: SwitchMem dynamically routes between textual and visual memory representations at each interaction step.
- Results: it outperforms static-memory baselines across three GUI benchmarks, supporting the claim that memory modality is a step-dependent decision rather than a fixed design choice.
Code and project resources are open-sourced.
More from coding & agent
- Codex autocompaction gets stuck: one steer and 10 million tokens later it's still parroting stale state — Aryvyo · 2026-10-14
- Agentic engineering vs AI engineering: Hugo Bowne-Anderson's clean split — Al_Grigor · 2026-10-14
- PostHog to devtools: stop asking 'how to add AI,' ask 'what's the agent-first version' — edwin · 2026-10-14
- Devin SWE-2 jumps to #3 on attention-to-detail coding test, tying Claude Sonnet 5.5 — silasalberti · 2026-10-14
- MCP hits ~500M monthly downloads but stays free — the money goes to key-management security firms — FinanceYF5 · 2026-10-14
- One-shot flying dragon: Gemini 4 Argon with Blender MCP and Tripo 3D builds a pure .blend scene — sidahuj · 2026-10-14