"MoE is a datacenter architecture; looped transformers are a laptop architecture"
andreisavu · x · 2026-11-22
@zhengyaojiang argues MoE trades memory for compute — huge weights with only a few active per token, which suits big batches and big GPUs, while laptops run batch size 1 with little memory. Looped transformers make the opposite trade: reuse the same weights, spend more compute, need far less memory.
He won't be surprised if local and cloud models end up with very different architectures, including ones invented just for local inference. The thread cites thinkymachines' Inkling 2 Mini, a 26B-parameter looped transformer claimed to run on a laptop and handle agentic work, scoring 20.1 on AA Index.
More from Infra
- WSJ: Associate of Jensen Huang's Partner Accused of Funneling Nvidia Chips to China — GaryMarcus · 2026-11-22
- Hierarchos-Native v0.2.5 ships P3 FP32-merge for RDNA3 WMMA GEMM, up to 27% faster — PhysicsDisastrous462 · 2026-11-22
- prime-rl 0.11.0 ships: 1.5x faster training, new pretraining entrypoint, Mega MoE dispatcher — samsja19 · 2026-11-22
- Math suggests 7.7M GPUs would serve everyone: fuel for the AI overbuild hypothesis — bookwormengr · 2026-11-22
- Chinese NAND maker YMTC promotes ZHITAI SSDs at Korea's G-Star game show — terryyuezhuo · 2026-11-22
- SGLang runs Inkling 2 Mini in 65GB on DGX Spark, 2x faster prefill than vLLM — AIFlow_ML · 2026-11-22