Attribution scores can silently mislead: gap check flags Captum error ~1 vs Tangermeme at 10⁻⁷
bravo_abad · x · 2026-10-10
Genomic models highlight stretches of DNA as important, and researchers may use those scores to pick lab experiments — but the software computing them can silently mishandle parts of the model.
- Schreiber documented the issue in his Tangermeme paper: some DeepLIFT/SHAP implementations return incorrect scores on unsupported operations.
- A useful check: per-base contributions should sum to the difference between the sequence's prediction and a reference sequence's; the unexplained gap should be near zero.
- In one custom model output, the gap was roughly 1 with Captum versus 10⁻⁷ with Tangermeme, which explicitly handles the operation.
Check the AI explanation before choosing the next experiment.
More from Research
- Google DeepMind launches AI co-clinician initiative for 'triadic care' in medicine — alan_karthi · 2026-10-10
- Harvard Medical School and Google DeepMind team up on conversational AI diagnostic study — alan_karthi · 2026-10-10
- DeepMind's AMIE diagnostic AI passes first real-clinic prospective test, published in The Lancet — alan_karthi · 2026-10-10
- Anthropic's Geoffrey Irving: is the formalized classification of finite simple groups coming next week? — geoffreyirving · 2026-10-10
- Simons Institute launches Fall '27 program on diffusions and flows, postdoc call open — gautamcgoel · 2026-10-10
- Cell-rejuvenating therapy shows early vision improvement in 2 of 3 glaucoma patients — kimmonismus · 2026-10-10