Community Notes debunk claim about MoE distant-token expert routing

A community note debunked the claim that distant tokens in MoE models never share experts, in a dispute relayed by Nathan Young citing Goodfire AI's Nick Cammarata; the Common Notes browser extension overlays such notes on any article or video.

2026-10-24 ~ 2026-10-24 · 2 related posts