Community math: MiniMax model's 512K context needs ~104GB KV cache, multi-GPU only
AIFlow_ML · x · 2026-10-26
AIFlowML estimated KV cache memory from MiniMax's model config (8 KV heads, headdim 128, BF16): 6.5GB at 32K tokens, 26GB at 128K, and 104GB at the full 512K context — meaning the marketed 512K context is a multi-GPU affair. Weights: 66GB in BF16; community quantizations Q4KM GGUF 20.8GB, Q5KM 25GB, Q80 35GB, so a single 24GB card can hold Q4 weights with room for short context.
More from Infra
- Anthropic engineer: Claude Code cache hit rates are now high, savings live — teortaxesTex · 2026-10-26
- Deco decouples 'thinking' from 'spelling' in generative recommenders for 3.2-8.6x throughput — _reachsumit · 2026-10-26
- Redditor with $4,000-$5,000 budget seeks local AI video generation hardware advice — dewsphere · 2026-10-26
- ThousandML Open-Sources CQ 4-bit QLoRA: 17.4% Lower Val Loss Than Unsloth NF4 — textclf · 2026-10-26
- Why OpenAI and Anthropic Cut LLM Prices by Up to 80% in Months — Azer_pubes33 · 2026-10-26
- GPU rental rates split by chip generation: B300 +8.6% in a week, A100 only +2% — AccBalanced · 2026-10-26