Single RTX 4090 runs 200k-context Qwen3.8 quant at 142 tok/s after three-flag Strata tune

sisyphus-cycle · reddit · 2026-10-26

Setup: One RTX 4090 (24GB), 48GB RAM, Ryzen 3900X, Windows; running Qwen3.8-Flash-Next IQ3S quant locally on the Strata inference engine v0.43 with 200k context.
Method: Had the model tune itself — web research plus sub-agents reading Strata's source. Author wrote a custom pi sub-agent extension that saves KV state before each sub-agent call via Strata mechanisms, reaching 450k total computed context without compaction.
Results: Grid-search-style tuning lifted decode from 97 to 142 tok/s (+45%) with just three flags: --adapt-async 1, --kv-resident 20480, STRATAADAPTLAG=2. Measured 2076 PP and 95-105 tok/s average on a 145k-context full reprocess, peaking at 145.
Negative optimizations: --expert-cache is in MiB not slots (−72% if you fill the card), --ple-io ram (−90% on 48GB), --no-prefill-borrow (−46%).
Bottleneck: Windows doesn't pin memory with CUDA, so host→GPU expert copies stay CPU-driven.
Full JSON config shared for reproduction.

More from coding & agent