Simple neg-log-odds loss swap improves every training attribution method tested
davidbau · x · 2026-10-31
davidbau highlights new training-attribution findings from his group: swapping in a negative-log-odds loss — more effective where the model is saturated — improved every attribution method they tried. Validation on FTRACE-TREx removes each method's top-ranked documents and retrains; with the fixes, accuracy drops on those facts are substantially larger for all methods, confirming better attribution fidelity.
Related event: Negative Log Odds Loss Boosts LM Training Data Attribution(2 posts)→
More from Research
- 4B decision model trained on 117k examples matches Jev — except it can't detect sarcasm — SRSchmidgall · 2026-10-31
- David Baker Lab Preprint: Protein Design Cuts CD8⁺ T Cell Immunogenicity — gottapatchemall · 2026-10-31
- Companies pay $300 per paper to survey ICLR/NeurIPS authors — grads can cash in — Icy-Park4287 · 2026-10-31
- Gary Marcus mocks OpenAI's new algorithm: exponential subroutine below a 10^25-variable threshold — AlexTensor · 2026-10-31
- After NVIDIA's BindClaw, every AI4Bio paper now needs an agentic pipeline demo to prove versatility — rishabh16_ · 2026-10-31
- Neural Thickets Revisited: Frozen-Model Logit Perturbations Match Full-Weight Perturbation Ensembles — kastnerkyle · 2026-10-31