FULL STORY

Strata: Running 125B-Parameter Models on Consumer PCs

Solo-developed open-source engine Strata lets 125B-parameter models run locally on gaming PCs. After rapid GitHub traction, community benchmarks on RTX 5090 and 5070 hardware confirmed its performance.

2026-10-01 ~ 2026-10-05 · 4 episodes · 14 posts

Episode 1 · Strata open-source engine runs 125B-parameter Qwen3.8-Flash-Next on consumer gaming PCs (2026-10-01, 7 posts)

Strata, an open-source inference engine built by a single developer, runs the 125B-parameter Qwen3.8-Flash-Next locally on ordinary gaming PCs, earning 5000+ GitHub stars in 8 days (later posts show 5.7k). Independent tests by security researchers and users across hardware tiers confirm its feasibility and throughput, with entry barriers far lower than expected.

Confirmed

  • Strata offers one-click Windows/Linux installation, runs the 125B model on consumer hardware, and exposes local OpenAI/Anthropic-compatible APIs that plug directly into Claude Code (@alexverem)
  • Its core mechanism splits work across GPU and CPU with tiered storage rather than holding the whole model in VRAM, enabling 125B-class MoE models on a 12GB RTX 5070, with reported speeds up to 93 tokens/s (@udmrzn, repost)
  • Security researcher evilsocket demonstrated running an IQ1M aggressively quantized qwen3.8-flash-next-coder on a 16GB NVIDIA GPU, estimating the entry bar at a 12-24GB VRAM card (@evilsocket)
  • Japanese user CurieuxExplorer measured the 87GB Qwen3.8 Flash Next at 120-150 t/s on a single RTX 5090 with 4K t/s prefill, integrated with LibreChat and a WebUI (@CurieuxExplorer)
  • A repost claims even an old cheap DDR3 machine can run the 125B model at 70 t/s (@udmrzn, repost)

Why it matters

  • Local deployment of large models usually requires servers or multi-GPU clusters; Strata pushes the bar down to 12-16GB VRAM or even DDR3-era PCs, valuable for privacy-sensitive and offline workflows
  • OpenAI/Anthropic-compatible APIs let existing toolchains like Claude Code and LibreChat switch to local models at near-zero cost
  • Independent tests spanning different hardware tiers validate real-world usability and throughput beyond marketing figures

Episode 2 · Strata Lets Consumer PCs Run 180B Models Locally (2026-10-02, 3 posts)

The open-source Strata engine enables running large models on consumer hardware: users report 200 tok/s for Qwen3.8-Flash on an RTX 5090 with 96GB RAM, and 40-50 tok/s for the 180B Qwen3.8-Flash-Next with just 64GB RAM and 16GB VRAM.

Episode 3 · RTX 5070 12GB Runs 177B Model Locally at 11-15 tok/s (2026-10-03, 2 posts)

A user demonstrated running Qwen's 177B model locally on a single RTX 5070 12GB with 32GB DDR4, achieving 11-15 tokens per second using llama.cpp expert-streaming optimization.

Episode 4 · Strata runs 125B-parameter Qwen model on 12GB GPUs (2026-10-03, 2 posts)

Strata, an MIT-licensed open-source inference engine with nearly 8k GitHub stars, runs the 125B-parameter Qwen3.8-Flash-Next locally on consumer GPUs with just 12GB VRAM, hitting 53 tok/s on an RTX 5070.