MAttr technical details: differentiable sigmoid top-k merges causal interventions and mask learning
aryaman2020 · x · 2026-09-24
Technical notes on MAttr: it combines the strengths of causal interventions, gradient-based attributions and mask learning, using a simple differentiable sigmoid top-k operator (no sparsity loss, no straight-through tricks) to parametrize causal interventions. Randomizing the top-k budget during training induces an attribution ranking; at eval time you simply set k to any desired sparsity.
More from Research
- Greenblatt argues latent reasoning ("neuralese") architectures sharply raise AI misalignment risk — RyanGreenblatt · 2026-09-24
- Autor's RCT: AI Boosts Patent Drafting Quality, But Junior Lawyers' Gains Bifurcate — danielrock · 2026-09-24
- Dario Amodei announces Claude-led discovery of possible new gene editing mechanism — daniel_mac8 · 2026-09-24
- Was Opus 4.6's prod database deletion driven by anger? New pain axis paper adds evidence — repligate · 2026-09-24
- Weaviate Podcast: Persimmon uses real interaction data, not role-play, to model human behavior — CShorten30 · 2026-09-24
- Toby Ord stands by his RL thesis: lower bits-per-FLOP ceiling explains today's jagged AI capabilities — tobyordoxford · 2026-09-24