"MoE is a datacenter architecture; looped transformers are a laptop architecture"

andreisavu · x · 2026-11-22

@zhengyaojiang argues MoE trades memory for compute — huge weights with only a few active per token, which suits big batches and big GPUs, while laptops run batch size 1 with little memory. Looped transformers make the opposite trade: reuse the same weights, spend more compute, need far less memory.

He won't be surprised if local and cloud models end up with very different architectures, including ones invented just for local inference. The thread cites thinkymachines' Inkling 2 Mini, a 26B-parameter looped transformer claimed to run on a laptop and handle agentic work, scoring 20.1 on AA Index.

Original post →

More from Infra

Infra channel →