MIT-licensed Strata runs Qwen3.8-Flash-Next (125B) locally at 53 tok/s on an RTX 5070 12GB

petrusenko_max · x · 2026-10-05

Strata, an MIT-licensed inference engine, runs Qwen3.8-Flash-Next (125B) locally with one-click Windows/Linux installs and an OpenAI/Anthropic-compatible API on localhost. Per its README, it measures 53 tok/s on an RTX 5070 12GB (IQ3S quant), 94 tok/s at Q20, with a 128K-context demo.

Related event: Strata runs 125B-parameter Qwen model on 12GB GPUs(2 posts)→

Original post →

More from Infra

Infra channel →