Permutation importance tells you about the model, not the physics: pitfalls explained
bravo_abad · x · 2026-10-27
A practical thread on permutation importance for scientific ML: shuffle one feature column, re-score, and error increase = importance. Two key pitfalls: redundant features (composition-weighted descriptors overlap heavily) can all look unimportant, while shuffling one of two correlated descriptors creates unrealistic inputs that inflate scores—group permutations help. Core principle: feature importance is a property of the model's response to a perturbation, not a claim about physical causality. Essential distinction when using ML for scientific insight.
More from Research
- Video demo: building verifiable scientific models with Lanyon AI — JohnCabot · 2026-10-28
- OptiMer Wins EMNLP2026 Outstanding Paper: Post-Hoc Data Mixing With 15-35x Lower Search Cost — prajdabre · 2026-10-28
- Paper: Fine-tuned Rerankers Beat Decision Models by 5–29 nDCG Points on Every Backbone Up to 9B — omarsar0 · 2026-10-28
- Citrini: Claude designed scorpion antivenom in 64 hours — monkeys are the next bottleneck — citrini · 2026-10-28
- Flow matching ported to the browser via ONNX and JavaScript, runs locally — NoenD_i0 · 2026-10-28
- Rebuilding 'Democracy Does Cause Growth': long-run GDP effect rises to 33.6% with 2024 data — ChenhaoTan · 2026-10-28