Strata engine runs Qwen3.8 Flash Next on a 12GB laptop at 51 t/s, 1500 t/s prefill
MLDataScientist · reddit · 2026-09-30
A hands-on test of the new Strata inference engine on a 12GB VRAM laptop (5070ti, 64GB RAM) running ISTA-DASLab's Qwen3.8 Flash Next GGUF: 51 t/s generation at 43k context (vs 23 t/s on stock llama.cpp with the same quant) and 1500 t/s prefill for 32k context (vs 100 t/s), with 11GB VRAM used. Caveats: Strata currently supports only this model and select ISTA-DASLab quants (IQ3S claimed to recover full coding performance), NVIDIA-only with experimental AMD support. Early kv-cache bugs are fixed.
More from Infra
- BIS issues rule blocking remote access to US AI chips via third-country data centers — teortaxesTex · 2026-11-16
- Qdrant 1.20.1: 50% fewer cold-storage reads, ~2x faster quantization, agent-ready setup — qdrant_engine · 2026-11-16
- One 16GB GPU runs songs, voice, images, 720p video and local RAG — all offline and open source — llama-of-death · 2026-11-16
- Starlink Now Protects Greek Trains From Collisions With 5cm-Accurate Tracking of 134 Locomotives — XFreeze · 2026-11-16
- Google donates Agent Substrate, gVisor and OpenRL to CNCF; Agent Substrate 1.0 goes GA on GKE — dl_weekly · 2026-11-16
- Nvidia scandal isn't slowing down: Huang's near-$100B power web under strain — The Verge AI · 2026-11-16