SGLang runs Inkling 2 Mini in 65GB on DGX Spark, 2x faster prefill than vLLM
AIFlow_ML · x · 2026-11-22
Benchmarks on a DGX Spark show SGLang serving Inkling 2 Mini in just 65GB of unified memory vs 105GB for vLLM, with 15.6k-token prompt prefill at 4.48s vs 9.23s. The 40GB headroom leaves room to run agents and tools alongside the model.
More from Infra
- WSJ: Associate of Jensen Huang's Partner Accused of Funneling Nvidia Chips to China — GaryMarcus · 2026-11-22
- Hierarchos-Native v0.2.5 ships P3 FP32-merge for RDNA3 WMMA GEMM, up to 27% faster — PhysicsDisastrous462 · 2026-11-22
- prime-rl 0.11.0 ships: 1.5x faster training, new pretraining entrypoint, Mega MoE dispatcher — samsja19 · 2026-11-22
- Math suggests 7.7M GPUs would serve everyone: fuel for the AI overbuild hypothesis — bookwormengr · 2026-11-22
- Chinese NAND maker YMTC promotes ZHITAI SSDs at Korea's G-Star game show — terryyuezhuo · 2026-11-22
- Open-source Agent Skill hill-climbs local LLM serving speed, users report up to 5x gains — CatAstro_Piyush · 2026-11-22