Running Ling 3.1 Flash (560B MoE, 25B Active) Across 4 DGX Sparks via llama.cpp RPC
BerryBlush_Rogers691 · reddit · 2026-10-18
User sudoing loaded Ling 3.1 Flash — a 560B MoE with 25B active parameters — across four NVIDIA DGX Sparks borrowed from a friend, using Atomic Chat's day-one Q4 GGUF via llama.cpp RPC since no official FP4 vLLM build existed. He expects bandwidth to be painful and plans to benchmark it against the 124B Ling 3.0 Flash.
More from Infra
- Auction-based LLM routing paper finds fixed token pricing wastes 10%-71% in costs — _onionesque · 2026-10-18
- SGLang Summit sponsors jump from 11 to 36 in one week — ying11231 · 2026-10-18
- Oracle CDS hits record 300bps, called first AI trade domino after Barclays sell downgrade — arthurcolle · 2026-10-18
- Agent Coding Workflow Would Cost ~$40k/Month on API Pricing, Dev Reveals — ChrisRackauckas · 2026-10-18
- PS5LM: llama.cpp port turns a jailbroken PS5 into a home LLM server (500+ GitHub stars) — wateriscoding · 2026-10-18
- DFB laser heterogeneously coupled to LNOI: 170 mW output, 45 mA threshold — jwt0625 · 2026-10-18