MIT-licensed Strata runs Qwen3.8-Flash-Next (125B) locally at 53 tok/s on an RTX 5070 12GB
petrusenko_max · x · 2026-10-05
Strata, an MIT-licensed inference engine, runs Qwen3.8-Flash-Next (125B) locally with one-click Windows/Linux installs and an OpenAI/Anthropic-compatible API on localhost. Per its README, it measures 53 tok/s on an RTX 5070 12GB (IQ3S quant), 94 tok/s at Q20, with a 128K-context demo.
Related event: Strata runs 125B-parameter Qwen model on 12GB GPUs(2 posts)→
More from Infra
- Thailand data center association cites $18B in GPU/hardware cancellations after SE Asia lending halt — SumitGup · 2026-11-19
- Gridline launches one-API fine-tuning, distillation and batch inference for open models — Scobleizer · 2026-11-19
- Ascend output could hit ~20M chips next year; memory and packaging, not logic, are the bottleneck — teortaxesTex · 2026-11-19
- AMD Loom boosts Qwen3.5-4B prefill up to 2.6x on Radeon 890M laptops — jfowers_amd · 2026-11-19
- Dev submits PR adding periodic checkpoints to GKE sandboxes so agents don't lose progress — yoimnotkesku · 2026-11-19
- AWS Lambda Adds GPU Support via Managed Instances for Inference and Media Workloads — DavidWells · 2026-11-19