Transformers PR streams MoE experts from SSD to run a 180B model on a 32GB Mac

jomic01 · reddit · 2026-11-20

Transformers PR 50717 adds an expertsoffload option that keeps MoE expert weights on SSD and loads only the experts the router selects.

Source: Hugging Face blog moe-expert-streaming.

Original post →

More from Infra

Infra channel →