Practitioner Laments: MoEs Were a Mistake and a Nightmare to Train
qtnx_ · x · 2026-09-17
An AI practitioner argues that Mixture-of-Experts (MoE) architectures "were a mistake" and are "such a nightmare to train" — a first-hand gripe about MoE training costs that runs against the industry-wide bet on MoE.
More from Models
- "Direct confidence readout" claim debunked: it's just entropy from the logit distribution — mgostIH · 2026-09-17
- Claude flags Kali Linux install as cyber, sparking over-refusal backlash — TinfoilTricorn · 2026-09-17
- Fixing one simple grader doubled the score—and the rest were errors in the tasks — xeophon · 2026-09-17
- Claude flagged a routine Kali Linux install as cyber, sparking overrefusal complaints — TinfoilTricorn · 2026-09-17
- Rumor: Gemini 4.0 Pro spotted testing in LMArena under the name Gemini 3.8 Flash — gaganghotra_ · 2026-09-17
- $200 plans are just a preview: analyst predicts frontier models will go API-only — StewartalsopIII · 2026-09-17