220B MoE with full 262K context runs on RTX 3090s: 251 tok/s on 4 cards
QuixiAI · x · 2026-11-20
Developer superalesha built maya-hy4f, an engine on Project Maya kernels that runs Tencent's Hy4-flash — a 220B MoE (512 experts/layer, 16 active) with full 262K context, vision, and MTP speculative decoding — on consumer hardware, now uncensored.
- 1x RTX 3090 + 64GB RAM: hot experts in VRAM, CPU computes the rest; IQ2 + NVMe needs only 45 GiB RAM, 54-68 tok/s
- 2x 3090: 70-123 tok/s
- 4x 3090 (all weights in VRAM): 176-251 tok/s; 223 tok/s average across a real 50K-token coding agent session, still 159-197 tok/s near 262K
- Q3K is reportedly near-lossless; vision encodes a 1080p image in 0.7-0.8s
No pruning: 1/2-card outputs agree with 4-card runs as much as two 4-card runs agree with each other.
More from Infra
- Transformers PR streams MoE experts from SSD to run a 180B model on a 32GB Mac — jomic01 · 2026-11-20
- Why IREN is down so bad: $15B capex vs $9B committed financing leaves $6B gap — tengyanAI · 2026-11-20
- Anthropic Gets Claude Prefill Running on AMD MI355X in Three Days — AccBalanced · 2026-11-20
- Customer cuts GitHub Actions spend 90%: 20-min job runs in 7 min on Celesto — aniketmaurya · 2026-11-20
- OpenSSL discloses CVSS 9.8 buffer overflow in EVP_EncodeBlock() — JacobHenner · 2026-11-20
- Anthropic's $2T valuation questioned: $104B net loss on $33B revenue in 9 months — deliprao · 2026-11-20