Dataset signatures propagate into user models and assistant evals, study finds
serinachang5 · x · 2026-10-08
Part two of the thread details the propagation path: HAI datasets → user models → assistant evals. Dataset choice affects user model outputs, evaluations of user model quality, and how a user model judges an LLM assistant — meaning conclusions built on a single dataset may carry source bias.
Related event: Study: dataset signatures contaminate user models and assistant evaluations(3 posts)→
More from Research
- EMNLP 2026 paper RECAP trains reasoning models to recover from unsafe trajectories — pinyuchenTW · 2026-10-08
- SaTML 2027 to host AdvML Frontiers workshop on human-centered trustworthy machine learning — pinyuchenTW · 2026-10-08
- OpenAI's new result proves 2005 edit-distance embedding optimal; researcher distills proof to 2.5 pages with AI help — thegautamkamath · 2026-10-08
- Experiments show smarter models and higher effort write better LLM-judge evals — danshipper · 2026-10-08
- Tencent's WorkForge scales verifiable training environments for long-horizon work agents — teortaxesTex · 2026-10-08
- Masked Geometric Encoder boosts 3D foundation models via frame dropping and self-distillation — zhenjun_zhao · 2026-10-08