GLM 5.2 runs at 35t/s via DwarfStar mixed RAM/VRAM inference
antirez · x · 2026-08-22
Antirez reports progress on the DGX Station: using DwarfStar mixed RAM/VRAM inference to run 4bit GLM 5.2 (500GB total weights). It achieves 35 tokens/s generation and 2000 tokens/s prefill (expected to hit 3k). The setup proves capable of running frontier open-weight models at agentic-usable speeds.
More from Infra
- NVIDIA releases Nemotron 3.5 Bolt 74B MoE that fits in 24GB VRAM at 2-bit — NerdyRodent · 2026-10-16
- Analyst: 80% of neoclouds may not survive, but the demand for their GPU capacity is real — BenBajarin · 2026-10-16
- Apple and Meta partner to run Muse Spark model on linked Mac Studios — pdamodaran · 2026-10-16
- Nvidia's developer base hits 11 million, up ~3.7x from 3 million in 2021 — Beth_Kindig · 2026-10-16
- Muse Spark Open runs locally on a Mac M3 Ultra via llama.cpp: frontier model on desktop — pcuenq · 2026-10-16
- Muse-Spark open-weights drop: 816B-A42B MoE model gets 359GB MLX 3-bit build — ns123abc · 2026-10-16