Strata runs a 125B model at 70tps on old DDR3 PCs with a cheap GPU
udmrzn · x · 2026-10-03
Strata (5.7k GitHub stars) is an open-source inference engine that runs the 125B-parameter Qwen3.8-Flash-Next on consumer hardware with one-click install for Windows/Linux. A cheap DDR3 machine with 64-128GB RAM plus a 8GB+ GPU hits 70+ tokens/s — more CPU cores means more speed. It exposes OpenAI/Anthropic-compatible local APIs, supports 128K context and image input, and can plug into coding agents with nothing leaving your machine.
More from Infra
- tinygrad lists $89,000 tinybox with 768GB VRAM: four refurbished AMD MI300X GPUs — viksit · 2026-11-19
- RustingStrata vs Strata on RTX 3060: Rust MoE engine edges out 28.3 vs 26.1 tok/s — Serious_Scarcity7818 · 2026-11-19
- Investor: market hates uncertainty more than money after Nvidia earnings — MartinGTobias · 2026-11-19
- SGLang ships Day-0 serving support for new model DaisyRose-27B — ying11231 · 2026-11-19
- Linux LTS patch flood: 3318 patches for 6.6 alone, 'no way humans are reviewing this' — pkqzy888 · 2026-11-19
- DaisyRose-27B's in-house sandbox: ~80% cheaper than cloud providers, 99.95% success rate — BanghuaZ · 2026-11-19