Hy team pushes back on ultra-sparse MoE: sparsity recovers efficiency, not quality

douglovefishing12 · reddit · 2026-11-22

From Hy's AMA: ultra-sparse MoE like 200B-A3B is inference-unfriendly on datacenter GPUs — it consumes memory without using FLOPs. Their scaling ladders show more sparsity helps at fixed active params, but cutting active params at fixed total (156B-A8B/A5B) should hurt quality a lot. The poster notes the trade-off may flip locally, where experts live in cheap RAM and compute is the bottleneck. Fun detail: DSA tipped the GQA-vs-MLA decision toward MLA, despite spec decoding becoming compute-bound.

Original post →

More from Infra

Infra channel →