CATT benchmark shows attribution methods fail to trace model answers to training data
cindy_x_wu · x · 2026-10-30
Researchers released CATT, a benchmark probing a counterintuitive failure: even when we know exactly which documents taught a model a fact, existing attribution methods often can't find them. CATT investigates why attribution fails and uncovers two simple fixes that improve tracing model answers back to their training data — relevant work for model transparency and data provenance research.
More from Research
- Persimmon team reveals internal learning evals were cut from the blog — gharik · 2026-10-31
- kalomaze's LoRA finetuning curves show overfitting typically starts at 3 epochs — kalomaze · 2026-10-31
- Stanford study: walking boosts creative thinking scores in 81% of participants — beechinour · 2026-10-31
- New monograph argues 20-watt neuromorphic AI can complement power-hungry transformers — ziv_ravid · 2026-10-31
- Study: none of today's SOTA latent world models faithfully model environment dynamics — udmrzn · 2026-10-31
- Fused Zip-NeRF: CUDA rewrite trains 6-7x faster, renders 10x faster on a 5090 — jon_barron · 2026-10-31