Zero-degradation Strata tuning: fixing layer_split lifts decode ~10-15% and halves CPU expert misses

PathfinderTactician · reddit · 2026-11-22

A Redditor shares months of Strata local-deployment tuning with a zero output-degradation constraint. Key points: use Alibaba-recommended sampling (temp 1.0, topp 0.95, topk 20); run "parallel": 2 — each Train slot costs only 0.4-0.6 GiB VRAM and the second Train reuses the first's KV cache, keeping decode busy while one Train tests; don't trust layersplit auto — measured expert coverage was 78% vs the 97.4% it assumes, and setting layersplit 16 cut CPU expert misses from 11.3% to 7.0% and lifted decode 8-13% across context depths (81.0→91.2 tok/s under 150K); lock experts in RAM with STRATANOLARGEPAGES=1 plus Windows 'Lock pages in memory'; add --prompt-cache-every 0 for full KV cache reuse; and a CUDA gradual slowdown after hours of runtime traces to expert cache filling VRAM (WDDM spilling to system RAM), fixable via custom build.

Original post →

More from coding & agent

coding & agent channel →