Study: classifiers can identify WildChat vs LMSYS chats from user messages alone
serinachang5 · x · 2026-10-08
New research led by Joseph J. Suh shows WildChat, LMSYS, and ShareChat don't paint the same picture of real-world AI use: a classifier can identify a chat's source dataset from user messages alone, and these dataset signatures propagate into user models and assistant evals, skewing conclusions downstream.
Related event: Study: dataset signatures contaminate user models and assistant evaluations(3 posts)→
More from Research
- KERNAUT uses coding agents and QD search to auto-discover interpretable kernel models — sirbayes · 2026-10-08
- Kernaut: coding agents design Gaussian process kernels via program search — sirbayes · 2026-10-08
- Kernaut's discovered kernel beats tuned standard kernels on glucose prediction — sirbayes · 2026-10-08
- Kernaut's discovered kernels stay interpretable: 16 scalar functions, inner-product form — sirbayes · 2026-10-08
- iOSWorld brings computer-use agent benchmarking to iOS at COLM 2026 — kohjingyu · 2026-10-08
- Epoch's InnovationEval: AI agents still far from producing real research innovations — Afinetheorem · 2026-10-08