Even after ablating the top MoE expert, models can still verbalize the new one
jonasgeiping · x · 2026-10-21
Jonas Geiping shares a robustness experiment from his team's new paper: ablating the top-1 expert in an MoE (which genuinely alters the forward pass) makes the task harder, yet trained models still correctly report the new top expert. Since the input text is unchanged and the difference is purely internal, this is a clean demonstration of trained 'digital proprioception'.
More from Research
- Seqlet caller follow-up: credits for DeepLIFT contributors — jmschreiber91 · 2026-10-21
- Center for AI Safety Publishes Full Syllabus for Intro to ML Safety Course — ArtificialOther · 2026-10-21
- Seqlet caller gets 22x speedup: 167,750 loci drop from 7.8s to 0.35s — jmschreiber91 · 2026-10-21
- X trends: SophontAI's Holon-S1 predicts gene activity from cancer slides; decision models rise — iScienceLuvr · 2026-10-21
- NEJM AI study: LLM-recommended care costs up to 10x more than physician orders — dvir_a · 2026-10-21
- Nucleotide dependency analysis turns genomic language models into functional element detectors — jmschreiber91 · 2026-10-21