Tencent releases GroupAssistBench: 50 group chats, 80K messages to test AI memory use
_akhaliq · x · 2026-11-10
Tencent released GroupAssistBench on Hugging Face, a real-world benchmark for memory in group-chat assistance.
- Scale: 50 group chats, 80K messages, 2,395 queries
- Tests whether assistants go beyond recalling past chats to actually using memories to help group members (e.g., reminding no-shows to pay up on departure day)
- Multi-level, multi-dimensional evaluation: recall for one user (L1) vs. group (M2) vs. with conflicts (M3), distinguishing memory recall from use; a point-based rubric penalizes answers grounded in superseded values
A useful eval set for teams building memory systems for chat assistants.
More from Research
- GraphRAG has a schema drift problem; Neo4j's ontology spec proposes a shared contract — JeremyCMorgan · 2026-11-11
- New philosophy paper: Moti Mizrahi replies to Nyholm on AI 'mental states' and longtermism — SvenNyholm · 2026-11-10
- UT Austin open-sources OopsieVerse, a damage-aware safety benchmark for household robots — DynamicWebPaige · 2026-11-10
- Scott Aaronson on AI solving centuries-old math problems, P vs NP debate — burny_tech · 2026-11-10
- Stanford releases new AI100 report on the Scaling Era and rise of generative AI — sarahookr · 2026-11-10
- Pre-norm breaks scale invariance: do modern Transformers invalidate a core DL theory assumption? — aaron_defazio · 2026-11-10