Two DGX Sparks run a 321B model with 819K-token context, cache survives restarts

AIFlow_ML · x · 2026-11-22

@greatYiTin runs Sage-S2.1-Flash on a pair of DGX Spark units at up to 37 tok/s with 0.16 s time-to-first-token and an 819K-token context window.

The standout detail: the prompt cache survives restarts. In long-context workloads, rebuilding a huge cache after every crash is where things get painful, so persistent caching makes local long-context inference far more practical.

Original post →

More from Infra

Infra channel →