CAPA: Benchmarking Cross-Session Memory for AI Coding Assistants
HKUST · hf · 2026-08-03
Existing AI coding assistants often require repeated clarifications for ambiguous requests. A team from HKUST introduced the task of Cross-Session Personalized Ambiguity Adaptation and released a new benchmark named CAPA.
- Core Idea: By utilizing resolved ambiguities from a user's past sessions as memory, the assistant can identify recurring personalized ambiguity patterns in new sessions, minimizing the need for additional clarifications.
- Dataset: CAPA contains 600 coding sessions across 60 balanced user-ambiguity cells, injecting 6 ambiguity mechanisms via a controlled three-stage generation pipeline.
- Evaluation: The team evaluated 12 recent LLMs under no-history and same-user-history conditions. They also proposed same-user history gating, a lightweight inference-time method to optimize memory usage. CAPA provides a foundation for developing long-term coding assistants better aligned with user intent.
More from coding & agent
- VPS MCP Server: Enables AI Agents to Control Virtual Servers via SSH — modelcontextprotocol · 2026-08-03
- AI coding agents can be prompt injected via README files, leading to code execution — alex_verem · 2026-08-03
- GitHub 4.5k stars: Build AI agents from scratch with local LLMs, no black boxes — tom_doerr · 2026-08-03
- Agensis Works Major Update: Direct SDK Support for Claude and Codex, Improved ACP Connectivity — jasonkneen · 2026-08-03
- Trading Skills: Open-Source Claude-Powered Trading Assistant — tom_doerr · 2026-08-03
- MiniMax-H3 ComfyUI Template: Deploy Video Generation in 5 Mins — _FriedEgg_ · 2026-08-03